Hi and Thanks again Professor Burk for all your great work trying to clarify this topic!... I continue my learning in spatial analysis. I have decided that in my case (Chilean economy), the k-nearest neighbor approach is the correct (to avoid bimodal distribution for the number of neighbors and to avoid islands given the difference in the size of the Chilean counties, and because I do not have data for all counties). I have constructed the W "standardized" matrix using STATA, Geoda, and R. All of them gave me different results. I have realized that Geoda does not row standardized by row, so it does not assign a 0.2 weight to each neighbor when I use, for instance, 5 nearest neighbors. Supposedly STATA and R give the same matrix, but I can not get the same results for my Moran's I test with both software. To create my standardized matrix in R I use the commands kneareigh, knn2nb and nb2listw and I have name the resulting matrix "k5w". Do you know how I can create a gwt file from my "k5w" object? You have suggested creating the W matrix in Geoda and using it in R or other software, but I want to do the opposite, that is, create the matrix in R and using it in Geoda (to take advantage of the global and local test in Geoda). Thanks again for any help!!!!
Hi Javier! The command write.nb.gal will take a ".nb" and write it as a ".gal" file for export to GeoDa. It is the last command described in the spdep manual. Just Google spdep and it will pop up for you to download. Good luck with it- I now how frustrating it can be trying to navigate and get the same results on different packages, especially when the code in many packages is still pretty raw (even in spdep, as wwe found with the bug!).
Thank you so much for sharing. This video is clear and informative, and it clarified my confusion in several spatial related issues. Thumb up particularly on spatial Hausman test section!!
Thank you for this intro to spatial econometric model. We are now enlightened to the meaning of these models. Now, we would like to see how it works when using panel data. respectfully
Thank you so much for the video! Just one update: the package spdep does not include anymore the lmSLX command. I had to download the package called spatialreg and in this one they have not fix the R^2.
Thanks for the update! Yes, they moved the regression commands from spdep into a new package, "spatialreg". Perhaps the package was getting too big? But, I really hate when packages do this, making all of your old code and documentation break, often for no good reason. I'll have to check the R^2 and let Roger Bivand know- or if you get a chance, you can. Just send the smallest snip of code that demonstrates the problem. I'll try to dig mine up, but it might be a couple of weeks since it is exam time. I am giving three, and taking one... the one I am taking looks like a beast (it is a take home, and I just got it).
Thank you for the most wonderful series in spatial econometrics. However I have a small question. Can you point me into the right direction (i.e. where to find the citations) regarding Luc Anselin's remark on the SARMA model being "probably not the right model"? Thanks a lot !
Thank you so much for helping to understand spatial econometrics (and to handle my master thesis)! One question that came to my mind while watching you lectures. Why are the p-values different between the outputs of the SLX model and the Impacts function? In my dataset this leads to different levels of significance for the variables. It would be great to get an answer!
If they are different, you must be adding the "R=500" part of the impacts command, but you shouldn't include it since simulations are not necessary for SLX models. If you include the R=, it will run simulations for estimating the p values, but if you leave it out you will get the exact p-values.
Thank you so much for such a fantastic video. I have one question: How can I generate equivalent models for SLX, Spatial Error, and Spatial Lag Models for spatial binary logistic regression models?
Are you asking "Can I include some variables with both lag and no-lag variables, but a few with only non-lagged?" If so, then yes, you can do this "by hand". There is a command that creates spatially lagged variables for you, and you can just add the lagged and non-lagged variables in a regular lm command using OLS.
Excellent video! One question I have is regarding the R value to determine the number of simulations... Is there a test we can run (such as a Power or Sample Size Test) that will allow us to see how many simulations (what value of R) we need to be certain the output of our model is valid or do we just have to trust that how ever many we select is good enough? Thanks!
Thanks for the explanation, professor. I have a question about one of the first steps of the process: Normally when doing an OLS, the procedure is (1) debug outliers, (2) make correlations, (3) use (preferably) the variables with a significant correlation to do the OLS. When looking to do a Spatial Regression, i shouldn't eliminate outliers anymore, right? Because that would mean removing some entities from my map and therefore it would no longer make sense to make a spatial weight matrix. I am right?
I don't agree with your procedures for OLS, but no one does, so I'll let that go. 😀 Yes removing outliers is going to be a problem especially with spatial models.
00:17:55 Hello, I would like to point to your attention the fact that the lmSLX() function is not in the {spdep} package, but in the {spatialreg} package instead.
Yes, they moved all of the regression commands to a new package right after I finished creating this course. Sadly, there is no way to post annotations on videos on UA-cam anymore to alert people.
Do you have a book you recommend for topics like this? Spatial analysis/regression, diagnostics/remedial measures, case studies, etc? I did my undergrad in mathematics, and I took some statistics classes (regression analysis, ANOVA/ANCOVA, fixed/mixed models, etc). I never took any spatial statistics, and I'd like to do some small projects aside from visualization.
@@BurkeyAcademy Thanks! By the way, thanks for these videos. There aren't many videos on spatial models out there, that I'm aware of. Most are non-spatial related.
Dear Professor I am trying to follow along your lecture, using your provided data and commands. However, i am unable to run the command used in line number 53 i.e. reg2 = lmSLX(reg.eq1, data = spat.data, listw1). Every time when i try to run this command i get the error message Error in lmSLX(reg.eq1, data = spat.data, listw1) : could not find function "lmSLX", I have spent much time browsing to identify the problem but could not find a solution. Can you please help in this regard?
Yes- They decided to split the regression functions out into a separate package, which really bothers me, since now everything is broken. You need to download the package called "spatialreg".
@@BurkeyAcademy Thank You very much for the reply. It was very helpful. I would like to thank you for the making and sharing such an informative, helpful and easy to understand video
Hi Professor, i think the error at 15:22 with the = "all" is caused by the quotation marks. The code used rabbit ears quotation marks (“ ”) when R only recognizes the straight line quotation marks (" "). There is a slight and subtle difference between the two sets. Hence the "all" was not highlighted green in the code. I hope that helps!
Thank you for your videos, it helped me a great deal! But I have a question I really need an answer to, so please if anyone could help... 1. I fitted multiple spatial regression models - spatial lag model, spatial error model and spatial durbin model in R. My question is, how do I check the assumptions of normality and homoscedasticity of errors? In classical linear regression, the diagnostics is done on standardized or studentized residuals, but how do I standardize residuals in these spatial models? When I use function "residuals()" to retrieve residuals from the spatial models and do QQplot, they always have this "S" shape. Does it mean the models are wrong? 2. The same for weighted spatial error model (using function errorsarlm with the argument weights =). Which residuals should I use to check the model assumptions?
In R I use the bptest.sarlm command. If you see an S shape in a QQplot, it doesn't mean your model is wrong, but it may mean that your residuals aren't normally distributed, which will affect the accuracy of p values (assuming you are looking at t he right residuals). However, this isn't something I'd personally obsess over, but just be cautious for borderline p values.
@@BurkeyAcademy I'm still wondering, I used maximum likelihood to estimate the parameters which is based on the normal distribution, so I'm still a bit unsure if the estimates would be correct when the assumption of normality is not met
Thank you very much for the explanation, its a great video! I got a doubt, though: when we talk about SAR, the direct effect is the one we see in the directly treated unit, the indirect effect is the one that spills over to the neighbors of the directly treated unit, and the total effect is the sum of the two. Given the feedback effect of SAR, does this mean that we will eventually observe the total effect in the directly treated units since only they experience both direct+indirect effect on them?
Page 37 in Lesage& Pace(2009) covers this. The indirect effect can both be interpreted as: 1) the total impact on all other regions from changing the variable in the home region, or 2) the total impact on the home region if ALL other areas changed their value of a variable in the same way.
@@BurkeyAcademy Exactly, but what is the interpretation of the total impact? Is it the overall impact experienced by the directly-treated units? (because they experience the direct treatment impact and the indirect feedback effect from their neighbors?..)
Absolutely! That one is coming next. I have everything ready to go, except no time right now! I promise, it won't be too long though (I hope!) I would be recording it today, but my microphone messed up on a video I have to make for a class I'm teaching, so I'll be re-recording that today instead. :(
31:40 hey, I noticed something with summary(impacts()). In my case I have a dataset with n=5000 and k=100, so because my knn matrix is so huge I use a trace for my summary(impacts()). When I do that, there is ZERO variation in the calculated parameter estimates, but the direct indirect and total p-values do still vary a bit (not major). I guess the lack of variation in parameter estimates is because I'm using a trace?
No- R's support for Spatial Panel Econometrics is still pretty sparse. I recommend Paul Elhorst's book which uses Matlab. However, I am horrible at Matlab.😐
Do you mean "Panel Data"? I don't have anything up yet, but two options to look at are the splm package in R; but a little more advanced tools are available in Matlab. Check out Paul Elhort's short book Spatial Econometrics: From Cross Sectional Data to Spatial Panels.
Yes. I am interested in analysis of spatial panel data. I just have to figure out how to analyze my data with GLMMs. Thanks for the reference. I really appreciate it!
In Matlab (code by James LeSage, Paul Elhorst, Don LaCombe, and others) they do maximum likelihood, plus lots of Bayesian stuff if you are into that. Here is a paper I wrote using their Matlab code: journal.srsa.org/ojs/index.php/RRS/article/view/229/204
Great video. Having trouble getting impacts for SEM and SDEM models in my dataset (SLX worked fine). Error in intImpacts(rho = rho, beta = beta, P = P, n = n, mu = mu, Sigma = Sigma, : length(listw$neighbours) == n is not TRUE.... Can this be because I have "islands" in my dataset? zero.policy=TRUE was used while running the models. How do I fix this?
For many, if not most spatial models, Islands simply aren't going to work. Delete them from the dataset, or choose a different definition of neighbors so that every observation has at least one neighbor.
@@BurkeyAcademy Many thanks for your reply. I had a feeling that was the case, but it's great to hear confirmation from the experts. Great content. Thank you for all the hard work you put into your videos and for replying to individual questions.
Dear professor, Thanks for all the amazing videos! they are helpful. I am trying to run my own data but I am getting the message 'Empty neighbour sets found' when I try to convert the .nb file to listw. Do you have any idea to help me with solving this problem?
Thank you professor for this wonderful explanation, I have one doubt, how will we perform the k fold cross validation in spatial models?, could you explain the validation of models in another video?
I am speaking mostly form ignorance here, but let me give it a shot: Spatial Econometrics is not Machine Learning, so we don't usually care about, nor usually have the opportunity to perform, something like out of sample validation (which I assume is something like what you are asking about). Since the data set is being treated as ONE large organism in our models, this doesn't make any sense to do. Our goal is to learn about patterns present in the data, measuring relationships and testing theories using the data we have. We are not trying to do predictive modeling, as such- it would never make sense to try to predict the value of an out of sample country, because WHERE this country is in the network is vitally important. There is going to be a constant strain between people who are trained and interested in only ML techniques, and people who are trained in statistics/econometrics. I don't really understand the point of most of what ML people do, because I just don't have the same goals in mind. I am going to try to learn the theoretical foundations of ML at some point soon, so I can try to understand things on "the other side". I am happy to discuss this further.
Hi Prof! I just want to ask something. Suppose i have a count data for my dependent variable Y and proportion data for some of my predictors (X's). Can i use those data for X's right away or do i need to sort of transform it to make it a count data? Thank you.
There are a couple of ways to go. Ideally, you want to be able to run Poisson and negative binomial models since your dependent variable is count. This is an example of a "limited dependent variable". The X's can be anything-- they are not as important as the form of the dependent variable. Unfortunately, I don't know of an ideal way to estimate spatial Poisson models. See my 2004 publication where I discuss this kind of model, and some things to try (especially footnote 16). Google "Burkey Simkins Payday", it is open access. The common method is to attempt a Bayesian solution, but I don't know much about that. I see that an R package "geoRglm" claims to make some progress with estimating spatial Poisson models.
does the neighbor lag coding only look at the influence of border sharing counties, or does it apply to all counties. I'm just wondering if you can set how exact neighboring counties and ones closer in area have an impact but ignore counties beyond a certain distance.
Yes, you can set neighbors any way you like (looks like you might be interested in distance threshold weights), and how the neighbors actually affect the model depends on the model (local versus global).
Thank you for your good videos. I have one problem. my problem is i program in python language and i want to implement this idea of spatial regression using the python language. Please can you help me with this work?
Thank you and i liked your whole work.The video explanation was very clear and superB!. All video resources well parked and shared, learners have all the resources at hand! However i ran into an error, can't install "spdep" package kindly help.I'm on the latest RStudio and run on Linux-Ubuntu 20.04. -Thank you
Hi professot! I'm running spatial regression using R but when I am installing the package 'regos' the system says that regos is not available for this version. I may wonder is regos a must for a spatial regression or do I have to upgrade my R software? Thanks
You only need the rgeos package if you need to extract centroids from a spatial data frame (and there are probably better ways to do it). However, if your R is that old, you might want to update it- it is free, after all! ☺
To cite me, or to cite the people originally developing these models? For me (and this video series) doi:10.18335/region.v2i2.98, for primary source material Anselin 1988 Spatial Econometrics: Methods and Models. Let me know if you need anything else!
Thank you for this ! But can you make a tutorial on how to create k-nearest neighbor distance weight matrix using shapefile? R keeps on telling me that the data (shapefile) should have coordinates.
This is definitely on my list to do. To make sure I can address your issue specifically, can you tell me more about your shapefile (or send it to me?) Fee free to email me mark at burkeyacademy dot com.
Hello Professor! upon reading a couple of articles it turns out I can create distance matrix thru geoda and just upload it in R. Although I have another question with regards to generating impact values on SLM model. Since my data has missing value, it seems that attributes from my regression model and my data are not identical. I've read that I should subset the data first before generating impact values, although I can't seem to find any suitable subsetting code for listw. any thoughts/ idea on this? Thank you!
Oh! I already have a video showing how to create weights matrices in GeoDa and another where I import them into R- see spatial.burkeyacademy.com. As for missing values, I have never tried to use a "dirty" data set with spatial data. I would get rid of all of the missing data, or fill in interpolated values as might be appropriate for your situation before starting the process. What do you mean by "SLM Model?"
I am an economist, and not familiar with this. However, if you send me some good sources, I can promise to try looking at it and see if it is something I could explore.
Hi and Thanks again Professor Burk for all your great work trying to clarify this topic!...
I continue my learning in spatial analysis. I have decided that in my case (Chilean economy), the k-nearest neighbor approach is the correct (to avoid bimodal distribution for the number of neighbors and to avoid islands given the difference in the size of the Chilean counties, and because I do not have data for all counties). I have constructed the W "standardized" matrix using STATA, Geoda, and R. All of them gave me different results. I have realized that Geoda does not row standardized by row, so it does not assign a 0.2 weight to each neighbor when I use, for instance, 5 nearest neighbors. Supposedly STATA and R give the same matrix, but I can not get the same results for my Moran's I test with both software. To create my standardized matrix in R I use the commands kneareigh, knn2nb and nb2listw and I have name the resulting matrix "k5w". Do you know how I can create a gwt file from my "k5w" object?
You have suggested creating the W matrix in Geoda and using it in R or other software, but I want to do the opposite, that is, create the matrix in R and using it in Geoda (to take advantage of the global and local test in Geoda).
Thanks again for any help!!!!
Hi Javier! The command write.nb.gal will take a ".nb" and write it as a ".gal" file for export to GeoDa. It is the last command described in the spdep manual. Just Google spdep and it will pop up for you to download. Good luck with it- I now how frustrating it can be trying to navigate and get the same results on different packages, especially when the code in many packages is still pretty raw (even in spdep, as wwe found with the bug!).
Fantastic!!! Such clear explanations and good constant reminders to be critical of your own results.
Thank you so much for sharing. This video is clear and informative, and it clarified my confusion in several spatial related issues. Thumb up particularly on spatial Hausman test section!!
Thanks for illustrating this! I've been following your channel for a long time and I really appreciate your explanation of everything :)
Fantastic course! Very clear and comprehensive. Thanks
Thank you a lot for your video, your explanation is really clear. It helps me a lot to learn spatial analysis for my thesis
Thank you for this intro to spatial econometric model. We are now enlightened to the meaning of these models.
Now, we would like to see how it works when using panel data.
respectfully
Very good! Thank you Dr. Mark!
Thank you very much from Brazil
Keep it up! this is amazing work!
Thank you so much for the video! Just one update: the package spdep does not include anymore the lmSLX command. I had to download the package called spatialreg and in this one they have not fix the R^2.
Thanks for the update! Yes, they moved the regression commands from spdep into a new package, "spatialreg". Perhaps the package was getting too big? But, I really hate when packages do this, making all of your old code and documentation break, often for no good reason. I'll have to check the R^2 and let Roger Bivand know- or if you get a chance, you can. Just send the smallest snip of code that demonstrates the problem. I'll try to dig mine up, but it might be a couple of weeks since it is exam time. I am giving three, and taking one... the one I am taking looks like a beast (it is a take home, and I just got it).
Thanks a lot for this pedagogic video, as the habit!
Thank you for sharing such great content!
Thank you for the most wonderful series in spatial econometrics. However I have a small question. Can you point me into the right direction (i.e. where to find the citations) regarding Luc Anselin's remark on the SARMA model being "probably not the right model"? Thanks a lot !
Thank you so much for helping to understand spatial econometrics (and to handle my master thesis)! One question that came to my mind while watching you lectures. Why are the p-values different between the outputs of the SLX model and the Impacts function? In my dataset this leads to different levels of significance for the variables. It would be great to get an answer!
If they are different, you must be adding the "R=500" part of the impacts command, but you shouldn't include it since simulations are not necessary for SLX models. If you include the R=, it will run simulations for estimating the p values, but if you leave it out you will get the exact p-values.
Amazing tutorial!
Congratulations!
The bug was fixed neatly.
This is just fantastic!
Thank u
You are welcome! Cheers!
Thank you for this video!
Thank you so much for such a fantastic video. I have one question: How can I generate equivalent models for SLX, Spatial Error, and Spatial Lag Models for spatial binary logistic regression models?
Amazing video!!!
Very good. However, I've tried to do a similar approach with point data without success. Any tips?
Very clear, thanks a lot !
Dear Mr Burkey, is there a way to include non-spatially lagged variables in the slx model?
Are you asking "Can I include some variables with both lag and no-lag variables, but a few with only non-lagged?" If so, then yes, you can do this "by hand". There is a command that creates spatially lagged variables for you, and you can just add the lagged and non-lagged variables in a regular lm command using OLS.
Thank you very much for your work. I have a question: would it be possible to perform a spatio-temporal regression using this same methodology?
Yes, people use this for panel and time series work (though I am not an expert on these). Chapter 7 of LeSage & Pace's book discusses this.
@@BurkeyAcademy Thank you so much!!!!
Great! Thanks!
Excellent video! One question I have is regarding the R value to determine the number of simulations... Is there a test we can run (such as a Power or Sample Size Test) that will allow us to see how many simulations (what value of R) we need to be certain the output of our model is valid or do we just have to trust that how ever many we select is good enough? Thanks!
Excellent thank you!
Such a good tutorial! You should start a patreon!
Hee hee, I see you figured it out! Thanks for becoming a Patron! ☺
Thanks for the explanation, professor. I have a question about one of the first steps of the process: Normally when doing an OLS, the procedure is (1) debug outliers, (2) make correlations, (3) use (preferably) the variables with a significant correlation to do the OLS. When looking to do a Spatial Regression, i shouldn't eliminate outliers anymore, right? Because that would mean removing some entities from my map and therefore it would no longer make sense to make a spatial weight matrix. I am right?
I don't agree with your procedures for OLS, but no one does, so I'll let that go. 😀 Yes removing outliers is going to be a problem especially with spatial models.
00:17:55 Hello, I would like to point to your attention the fact that the lmSLX() function is not in the {spdep} package, but in the {spatialreg} package instead.
Yes, they moved all of the regression commands to a new package right after I finished creating this course. Sadly, there is no way to post annotations on videos on UA-cam anymore to alert people.
@@BurkeyAcademy Hopefully people get to read these comments. And thank you very much for the awesome playlist.
Do you have a book you recommend for topics like this? Spatial analysis/regression, diagnostics/remedial measures, case studies, etc?
I did my undergrad in mathematics, and I took some statistics classes (regression analysis, ANOVA/ANCOVA, fixed/mixed models, etc).
I never took any spatial statistics, and I'd like to do some small projects aside from visualization.
I recommend starting with J Paul Elhorst's
Spatial Econometrics : From Cross-Sectional Data to Spatial Panels
@@BurkeyAcademy Thanks! By the way, thanks for these videos. There aren't many videos on spatial models out there, that I'm aware of. Most are non-spatial related.
Dear Professor I am trying to follow along your lecture, using your provided data and commands. However, i am unable to run the command used in line number 53 i.e. reg2 = lmSLX(reg.eq1, data = spat.data, listw1). Every time when i try to run this command i get the error message Error in lmSLX(reg.eq1, data = spat.data, listw1) : could not find function "lmSLX", I have spent much time browsing to identify the problem but could not find a solution. Can you please help in this regard?
Yes- They decided to split the regression functions out into a separate package, which really bothers me, since now everything is broken. You need to download the package called "spatialreg".
@@BurkeyAcademy Thank You very much for the reply. It was very helpful. I would like to thank you for the making and sharing such an informative, helpful and easy to understand video
Hi Professor, i think the error at 15:22 with the = "all" is caused by the quotation marks. The code used rabbit ears quotation marks (“ ”) when R only recognizes the straight line quotation marks (" "). There is a slight and subtle difference between the two sets. Hence the "all" was not highlighted green in the code. I hope that helps!
You might be right...
@@BurkeyAcademy He's right.
How to deal with missing data? What if some areas (e.g. counties) do not have data at all?
Thank you for your videos, it helped me a great deal!
But I have a question I really need an answer to, so please if anyone could help...
1. I fitted multiple spatial regression models - spatial lag model, spatial error model and spatial durbin model in R. My question is, how do I check the assumptions of normality and homoscedasticity of errors? In classical linear regression, the diagnostics is done on standardized or studentized residuals, but how do I standardize residuals in these spatial models? When I use function "residuals()" to retrieve residuals from the spatial models and do QQplot, they always have this "S" shape. Does it mean the models are wrong?
2. The same for weighted spatial error model (using function errorsarlm with the argument weights =). Which residuals should I use to check the model assumptions?
In R I use the bptest.sarlm command. If you see an S shape in a QQplot, it doesn't mean your model is wrong, but it may mean that your residuals aren't normally distributed, which will affect the accuracy of p values (assuming you are looking at t he right residuals). However, this isn't something I'd personally obsess over, but just be cautious for borderline p values.
@@BurkeyAcademy I'm still wondering, I used maximum likelihood to estimate the parameters which is based on the normal distribution, so I'm still a bit unsure if the estimates would be correct when the assumption of normality is not met
Well, I think an answer for that can be obtained by monte carlo simulation, just in case anyone ever cared.. :)
Thank you very much for the explanation, its a great video! I got a doubt, though: when we talk about SAR, the direct effect is the one we see in the directly treated unit, the indirect effect is the one that spills over to the neighbors of the directly treated unit, and the total effect is the sum of the two. Given the feedback effect of SAR, does this mean that we will eventually observe the total effect in the directly treated units since only they experience both direct+indirect effect on them?
Page 37 in Lesage& Pace(2009) covers this. The indirect effect can both be interpreted as: 1) the total impact on all other regions from changing the variable in the home region, or 2) the total impact on the home region if ALL other areas changed their value of a variable in the same way.
@@BurkeyAcademy Exactly, but what is the interpretation of the total impact? Is it the overall impact experienced by the directly-treated units? (because they experience the direct treatment impact and the indirect feedback effect from their neighbors?..)
Amazing!
Great tutorial thanks! Are you gonna explain the Durbin Spatial Model in R? Thank you very much.
Absolutely! That one is coming next. I have everything ready to go, except no time right now! I promise, it won't be too long though (I hope!) I would be recording it today, but my microphone messed up on a video I have to make for a class I'm teaching, so I'll be re-recording that today instead. :(
Please make a lecture on spatial structure of forest by nearest neighbor method
Thank you so much for this highly informative and practical lesson.
I was wondering if we could transform our data before we do the spatial analysis?
Yes, pretty much the same mathematical rules apply as do normal regression.
BurkeyAcademy Sounds great. Thanks for your reply.
31:40 hey, I noticed something with summary(impacts()). In my case I have a dataset with n=5000 and k=100, so because my knn matrix is so huge I use a trace for my summary(impacts()). When I do that, there is ZERO variation in the calculated parameter estimates, but the direct indirect and total p-values do still vary a bit (not major).
I guess the lack of variation in parameter estimates is because I'm using a trace?
Thank you. Can this be done on spatial count data? I want to model crash data at block group level.
I think that Bayesian methods for spatial Poisson are now fairly common, but I am not sure of an exact paper or source for you.
Hi Professor Burkey, your shp file looks like cross-sectional data. Did you upload R tutorial on panel data?
No- R's support for Spatial Panel Econometrics is still pretty sparse. I recommend Paul Elhorst's book which uses Matlab. However, I am horrible at Matlab.😐
@@BurkeyAcademy Thanks for your reply
Dr. Burkey, I tried to reach you at the address you posted, but it doesn´t work. How can I e-mail you?
Your videos are so helpful. Do you have any resources about how to do spatial regression for data with repeated measures?
Do you mean "Panel Data"? I don't have anything up yet, but two options to look at are the splm package in R; but a little more advanced tools are available in Matlab. Check out Paul Elhort's short book Spatial Econometrics: From Cross Sectional Data to Spatial Panels.
Yes. I am interested in analysis of spatial panel data. I just have to figure out how to analyze my data with GLMMs. Thanks for the reference. I really appreciate it!
In Matlab (code by James LeSage, Paul Elhorst, Don LaCombe, and others) they do maximum likelihood, plus lots of Bayesian stuff if you are into that. Here is a paper I wrote using their Matlab code: journal.srsa.org/ojs/index.php/RRS/article/view/229/204
Great video. Having trouble getting impacts for SEM and SDEM models in my dataset (SLX worked fine). Error in intImpacts(rho = rho, beta = beta, P = P, n = n, mu = mu, Sigma = Sigma, : length(listw$neighbours) == n is not TRUE.... Can this be because I have "islands" in my dataset? zero.policy=TRUE was used while running the models. How do I fix this?
For many, if not most spatial models, Islands simply aren't going to work. Delete them from the dataset, or choose a different definition of neighbors so that every observation has at least one neighbor.
@@BurkeyAcademy Many thanks for your reply. I had a feeling that was the case, but it's great to hear confirmation from the experts. Great content. Thank you for all the hard work you put into your videos and for replying to individual questions.
Dear professor, Thanks for all the amazing videos! they are helpful. I am trying to run my own data but I am getting the message 'Empty neighbour sets found' when I try to convert the .nb file to listw. Do you have any idea to help me with solving this problem?
Hi, my lmSLX does not work. kindly help. What am I doing wrong?
Without more information, there is no way to tell.
Thank you professor for this wonderful explanation, I have one doubt, how will we perform the k fold cross validation in spatial models?, could you explain the validation of models in another video?
I am speaking mostly form ignorance here, but let me give it a shot: Spatial Econometrics is not Machine Learning, so we don't usually care about, nor usually have the opportunity to perform, something like out of sample validation (which I assume is something like what you are asking about). Since the data set is being treated as ONE large organism in our models, this doesn't make any sense to do. Our goal is to learn about patterns present in the data, measuring relationships and testing theories using the data we have. We are not trying to do predictive modeling, as such- it would never make sense to try to predict the value of an out of sample country, because WHERE this country is in the network is vitally important. There is going to be a constant strain between people who are trained and interested in only ML techniques, and people who are trained in statistics/econometrics. I don't really understand the point of most of what ML people do, because I just don't have the same goals in mind. I am going to try to learn the theoretical foundations of ML at some point soon, so I can try to understand things on "the other side". I am happy to discuss this further.
@@BurkeyAcademy Thank you very much professor.
Hi Prof! I just want to ask something. Suppose i have a count data for my dependent variable Y and proportion data for some of my predictors (X's). Can i use those data for X's right away or do i need to sort of transform it to make it a count data? Thank you.
There are a couple of ways to go. Ideally, you want to be able to run Poisson and negative binomial models since your dependent variable is count. This is an example of a "limited dependent variable". The X's can be anything-- they are not as important as the form of the dependent variable. Unfortunately, I don't know of an ideal way to estimate spatial Poisson models. See my 2004 publication where I discuss this kind of model, and some things to try (especially footnote 16). Google "Burkey Simkins Payday", it is open access. The common method is to attempt a Bayesian solution, but I don't know much about that. I see that an R package "geoRglm" claims to make some progress with estimating spatial Poisson models.
Thank you so much! I appreciate your response and i will definitely look upon what you suggested. ☺️
Crimestat software allows to run spatial poisson regression analyses.
does the neighbor lag coding only look at the influence of border sharing counties, or does it apply to all counties. I'm just wondering if you can set how exact neighboring counties and ones closer in area have an impact but ignore counties beyond a certain distance.
Yes, you can set neighbors any way you like (looks like you might be interested in distance threshold weights), and how the neighbors actually affect the model depends on the model (local versus global).
Thank you for your good videos. I have one problem. my problem is i program in python language and i want to implement this idea of spatial regression using the python language. Please can you help me with this work?
Look up PySAL by Luc Anselin and his colleagues.
Sir, Thank u for this video and please upload the video for conditional autoregressive model in R Software which is helpful to me and many people's...
Thank you and i liked your whole work.The video explanation was very clear and superB!.
All video resources well parked and shared, learners have all the resources at hand!
However i ran into an error, can't install "spdep" package kindly help.I'm on the latest RStudio and run on Linux-Ubuntu 20.04.
-Thank you
If you tell me what the error is, I can try to help- but I'll just be Googling, since I don't have any special knowledge about it...
Hi professot! I'm running spatial regression using R but when I am installing the package 'regos' the system says that regos is not available for this version. I may wonder is regos a must for a spatial regression or do I have to upgrade my R software? Thanks
*professor
Sorry for the typo
You only need the rgeos package if you need to extract centroids from a spatial data frame (and there are probably better ways to do it). However, if your R is that old, you might want to update it- it is free, after all! ☺
@@BurkeyAcademy Okay, then I'll update it. Thank you professor! Fantastic illustration!
Thanks!
Very good explanation, helped me a lot, much better then my profs, can you give literature recommendations for cite?
To cite me, or to cite the people originally developing these models? For me (and this video series) doi:10.18335/region.v2i2.98, for primary source material Anselin 1988 Spatial Econometrics: Methods and Models. Let me know if you need anything else!
@@BurkeyAcademy thank you very much I will cite both. So I can cite your journal in the link, right?
@@MrProfTe Sorry, I sent the doi for the wrong one. Try this one instead: openjournals.wu-wien.ac.at/ojs/index.php/region/article/view/254
Thank you for this ! But can you make a tutorial on how to create k-nearest neighbor distance weight matrix using shapefile? R keeps on telling me that the data (shapefile) should have coordinates.
This is definitely on my list to do. To make sure I can address your issue specifically, can you tell me more about your shapefile (or send it to me?) Fee free to email me mark at burkeyacademy dot com.
Hello Professor! upon reading a couple of articles it turns out I can create distance matrix thru geoda and just upload it in R.
Although I have another question with regards to generating impact values on SLM model.
Since my data has missing value, it seems that attributes from my regression model and my data are not identical. I've read that I should subset the data first before generating impact values, although I can't seem to find any suitable subsetting code for listw. any thoughts/ idea on this? Thank you!
Oh! I already have a video showing how to create weights matrices in GeoDa and another where I import them into R- see spatial.burkeyacademy.com. As for missing values, I have never tried to use a "dirty" data set with spatial data. I would get rid of all of the missing data, or fill in interpolated values as might be appropriate for your situation before starting the process. What do you mean by "SLM Model?"
Good tutorials, could you please do some on Species distribution models
I am an economist, and not familiar with this. However, if you send me some good sources, I can promise to try looking at it and see if it is something I could explore.
Thanks!