@@burdGISyeah, I have a lot of issues, right now i am trying to iterate through a bunch of tifs that are in geodetic to project them all and then resample them. The model starts working but before finishing detects a problem with the datum transformation and its not running anymore. This is really weird because all the layers have the same lat long +wgs84 model. I am just baffled how tricky and capricious the software can be any idea?
I take it you're using Arc. I'd be happy to have a look at it if you want to send me a sample of the data and your model (plus exactly what you need to do). I have a couple of models already set up to do batch processing for rasters. Both reprojecting and resampling. You can send any data via the burdgis email (just info at burdgis dot com). Let me know which version of Arc you're using too.
I am having an issue connecting the iterated NetCDF files as you did in 10:50 to the "Make NetCDF Raster Layer" Tool; I get an error that the input layer is not multidimensional. But when I manually select a file from the directory as input to the NetCDF raster tool, all works perfectly and it does treat the one netcdf as multidimensional. Any idea why I get this error?
hello sir, do you still answer question for this tutorial. i followed your instruction and it was so helpful, so now how i can copy raster from file goedatabase as a Tiff image to another folder to use it for analysis?
@@burdGIS Sir, i am using ArcMap. I was not able to batch it in Model builder to export the image as Tiff format. it will be so nice of you if you could edit the model and add another export option as exporting converted netcdf to Tiff. Thank you so much and how i can write you personally to ask you about trainings.
@@burdGIS Here is the link - co2.jpl.nasa.gov/#mission=OCO-2 It's available in both NetCDF and HDF format. When I am opening the NetCDF file in ArcMap, it opens like a long vertical bar of raster even after changing variables.
Thanks for watching Oluwole. Please let me know which part is not clear. The exact step you are having difficulty with. It may help to watch the whole series as whatever part you are having difficulty with may be covered in earlier videos. You can find the full playlist here: ua-cam.com/video/lPEtfbXcOEw/v-deo.html
Hi Neha, thanks for the question. I've not had time to check this yet but I'm almost certain it will be due to the output type of the preceding tool or function. Model builder (and Arc geoprocessing tools in general) are very particular about output types and a feature class in a GDB is very different to a shapefile in a folder.
Hi, this video is really helpful, thank you By the way, can you help me? I'm working with .nc files too(Sentinel-5 nitrogen dioxide data) - basically I have to extract "bands" like lat, long, qa_value, and nitrogendioxide to a .csv file, then convert the CSV file to vector, then clip it to the borders of the country and extract values that have qa_value > 0.75. So far I have tried using SNAP to extract "bands" and ArcMap Model Builder to do the rest. The problem is SNAP writes raster width and resolution at the top of the table as comment lines (for example #RasterWidth=420) which makes the table unreadable for looking(because ArcMap doesn't see the column names 'lat' and 'long') and uses tabs instead of commas - both are really problematic because there's a 160 .nc files. Also, for some reason, SNAP doesn't want to open some of the .nc files. Today I tried the ArcMap model(your vid was really helpful as I didn't have an idea how iterators work) on a sample of 3 manually edited CSV files(deleted the # lines and replaced tabs with commas) and it worked perfectly, so it's really just a matter of extracting the values in a better way. I tried using ArcMap "Make Table View from NetCDF" but for some reason, when I tried using the "Attribute" dropdown, it was completely blank, as like ArcMap couldn't see the attributes in the .nc file If you can help me, that would be great, as I really don't know what I'm doing wrong
Hi Azzaz, I'm sure I could help with this. Please send me a mail with a sample of the raw data and I'll see what I can do. You can get me on info at burdgis dot com
Hello, can you make a tutorial for nested iterators? When you have for example 2 shapefiles ("A", "B") where you want to select from one and from another one to run a process that requires data from 2 sources. I have found poor examples on the matter. Thank you!
Thanks El Mono. Please could you describe the problem more clearly? Feel free to send a data sample with a description of the desired outcome. www.burdgis.com/contact
dear @@burdGIS! I wanted to clip the output image. for this it clipped the data to my vector shapefile but the pixcels are shown out of the study area also study area having missing pixcels. i also tried to extract by mask but the same issue i am facing
hello , can ı do iterations for dimensions instead of files also. for example ı have temperature data according time and depth. ı want to do operations between time or depth and they are all(temperature, time, lan, lot and time ) in a single netcdf file.
Dear burdGIS: Now I can tell you my challenge :). I have layers with 12 bands. This layers correspond to different variables (average temperature, humidity, etc.) Each band corresponds to a month. I want to apply different formulas to this bands in a Summer/Winter period. The problem is my Winter period that is from September-December of the current year + January and February of the next year. How to call this bands In a model to be applied in a 7 years period? I cannot figure it out.
Tiffs are limited to 32767 bands. So would creating an 84 band tiff work? Without seeing the data nor knowing exactly what you'd like to do another option would be to create two empty folders (summer and winter) and then extract the necessary bands (per year) to create new tiffs (one for each season each year). The outputs go into the corresponding folders with a name like "year01_summer". Then you can iterate through each folder applying the necessary formula. It may not be the most efficient approach and think ahead to any other processing you may wish to do. Let me know if that helps and I'd be happy to make a demo model is you can send 3 years worth of data or let me know where I can download similar data.
burdGIS I think I did not explain myself very well. sorry! So I will try my best: I have a bunch of raster (period 2011-2018) (.tif) that originally come from a netcdf file. This raster set contains information about different variables (temperature, rain, etc). Some of the raster come in a multiband format (12 bands, one per month) or in a single format (one band of a “x” month of the year) What I need to do is to create an iterative process in which these years are going to be transformed into a summer and winter. So instead of having 12 bands or months per variable per year, I will end up with 2: summer and winter per year. For example Summer 2011 = Avg(March,April,May,June,July,August) 2011 Winter 2011-2012 = Avg(September,October,November,December)(2011) & Avg(January,February)(2011+1) Summer 2012 = Avg(March,April,May,June,July,August) 2012 Winter 2012-2013 = Avg(September,October,November,December)(2012) & Avg(January,February)(2012+1) And so on, until cover the whole period (2011-2018) with 14 rasters . One site where you can download (but is daily data) is daac.ornl.gov/cgi-bin/dataset_lister.pl?p=32 Thanks again ;)
Hi, I followed your steps and I manage to make the model run with my .nc files. The problem is that the model runs all the files with the same name initials, but when it starts to read my other files in the same folder (with different name initials) it stops the process and gives me errors messages. I am really a beginner and have no idea why the model it is not able to iterate through the other files with different names. Could you help?
+Nathan S. Debortoli thanks for the detailed explanation of what is happening. Would you be able to email me a sample of your data so I could take a look? The model in the video is quite specific to the data from Ghana so it would be a fun challenge to write a more robust model.
Hi Shaban, you can get netcdf files from lots of different places. I think NASA have some, NOAA certainly will. You can also make your own with ArcGIS if you have suitable data. If you need any further help with netCDFs please let me know.
You have saved my life, it's working perfectly! Thank you so so much!
Thanks max. Glad to hear it's working for you
Thank you very much, your video is megafantastic
respect!!!. you just save me from 5 days of intensive repetitive work!!
Thank you mono! I hope you put the saved time to good use ;-) let me know of any other repetitive tasks come up! There's always a way
@@burdGISyeah, I have a lot of issues, right now i am trying to iterate through a bunch of tifs that are in geodetic to project them all and then resample them. The model starts working but before finishing detects a problem with the datum transformation and its not running anymore. This is really weird because all the layers have the same lat long +wgs84 model. I am just baffled how tricky and capricious the software can be any idea?
I take it you're using Arc. I'd be happy to have a look at it if you want to send me a sample of the data and your model (plus exactly what you need to do). I have a couple of models already set up to do batch processing for rasters. Both reprojecting and resampling. You can send any data via the burdgis email (just info at burdgis dot com). Let me know which version of Arc you're using too.
Hello i managed to do that, thanks to your videos. But don't worry, I still have a challenge for you ;). I will try to explain it clear later
I always appreciate a challenge ;-) Looking forward to it!
I am having an issue connecting the iterated NetCDF files as you did in 10:50 to the "Make NetCDF Raster Layer" Tool; I get an error that the input layer is not multidimensional. But when I manually select a file from the directory as input to the NetCDF raster tool, all works perfectly and it does treat the one netcdf as multidimensional. Any idea why I get this error?
Hi Chris, thanksfor watching. Where did the netcdf come from and can you share it? Also what version of the software are you using?
I have one netCDF file with different 50 years, how can I iterate and calculate averages
Where are the data from? Are they freely available? And, can you code? I'd probably go straight to Python if you're just pulling averages.
I can not run this model when i am adding %Name% in the clip output. how can i solve this?
Do you get an error message when you run it rashed? Feel free to send me more info via email.
hello sir, do you still answer question for this tutorial. i followed your instruction and it was so helpful, so now how i can copy raster from file goedatabase as a Tiff image to another folder to use it for analysis?
Hi What. Yes, you should just be able to right-click and export from the fgdb. Are using ArcPro or ArcMap?
@@burdGIS Sir, i am using ArcMap. I was not able to batch it in Model builder to export the image as Tiff format. it will be so nice of you if you could edit the model and add another export option as exporting converted netcdf to Tiff. Thank you so much and how i can write you personally to ask you about trainings.
You can contact me on info at burdgis dot com
Amazing video. It helped a lot.
Thanks neha. Glad you liked it. What project are you working on?
@@burdGIS Air Quality Analysis using remote sensing
@@burdGIS Do you know how to work with ISS OCO 2 level 2 carbon dioxide files? They are also in netcdf format but I am unable to open them in ArcGIS.
What happens when you try to open them Neha? Also do you have a link where I can get the data from in netcdf format?
@@burdGIS Here is the link - co2.jpl.nasa.gov/#mission=OCO-2
It's available in both NetCDF and HDF format. When I am opening the NetCDF file in ArcMap, it opens like a long vertical bar of raster even after changing variables.
Thanks, this is so educative, please the video was not clear at the beginning, what do l do pls, secondly how do l generate my netCDF data
Thanks for watching Oluwole. Please let me know which part is not clear. The exact step you are having difficulty with. It may help to watch the whole series as whatever part you are having difficulty with may be covered in earlier videos. You can find the full playlist here:
ua-cam.com/video/lPEtfbXcOEw/v-deo.html
tnx you bro, this video was great
Nice
Why does the model run only when we save outputs in gdb and not when we try to save them in a folder?
Hi Neha, thanks for the question. I've not had time to check this yet but I'm almost certain it will be due to the output type of the preceding tool or function. Model builder (and Arc geoprocessing tools in general) are very particular about output types and a feature class in a GDB is very different to a shapefile in a folder.
Hi, this video is really helpful, thank you
By the way, can you help me? I'm working with .nc files too(Sentinel-5 nitrogen dioxide data) - basically I have to extract "bands" like lat, long, qa_value, and nitrogendioxide to a .csv file, then convert the CSV file to vector, then clip it to the borders of the country and extract values that have qa_value > 0.75. So far I have tried using SNAP to extract "bands" and ArcMap Model Builder to do the rest. The problem is SNAP writes raster width and resolution at the top of the table as comment lines (for example #RasterWidth=420) which makes the table unreadable for looking(because ArcMap doesn't see the column names 'lat' and 'long') and uses tabs instead of commas - both are really problematic because there's a 160 .nc files. Also, for some reason, SNAP doesn't want to open some of the .nc files. Today I tried the ArcMap model(your vid was really helpful as I didn't have an idea how iterators work) on a sample of 3 manually edited CSV files(deleted the # lines and replaced tabs with commas) and it worked perfectly, so it's really just a matter of extracting the values in a better way. I tried using ArcMap "Make Table View from NetCDF" but for some reason, when I tried using the "Attribute" dropdown, it was completely blank, as like ArcMap couldn't see the attributes in the .nc file
If you can help me, that would be great, as I really don't know what I'm doing wrong
Hi Azzaz, I'm sure I could help with this. Please send me a mail with a sample of the raw data and I'll see what I can do. You can get me on info at burdgis dot com
Hello, can you make a tutorial for nested iterators? When you have for example 2 shapefiles ("A", "B") where you want to select from one and from another one to run a process that requires data from 2 sources. I have found poor examples on the matter. Thank you!
Thanks El Mono. Please could you describe the problem more clearly? Feel free to send a data sample with a description of the desired outcome. www.burdgis.com/contact
i am using the same procedure but my study area pixels are outside. Anysolution?
Thanks Aatika. What is the error message you are receiving?
dear @@burdGIS! I wanted to clip the output image. for this it clipped the data to my vector shapefile but the pixcels are shown out of the study area also study area having missing pixcels. i also tried to extract by mask but the same issue i am facing
What is the source data? Missing pixels in the study area sounds like NODATA values in the raster.
How to get the output clipped file in '.tif ' format in specific folder rather than .gdb???
Hi Santosh. Did you solve this?
hello ,
can ı do iterations for dimensions instead of files also. for example ı have temperature data according time and depth. ı want to do operations between time or depth and they are all(temperature, time, lan, lot and time ) in a single netcdf file.
I'd think so utku. What data source are you using? If you can provide a link to the data provider that would be great
Dear burdGIS: Now I can tell you my challenge :). I have layers with 12 bands. This layers correspond to different variables (average temperature, humidity, etc.) Each band corresponds to a month. I want to apply different formulas to this bands in a Summer/Winter period. The problem is my Winter period that is from September-December of the current year + January and February of the next year. How to call this bands In a model to be applied in a 7 years period? I cannot figure it out.
Tiffs are limited to 32767 bands. So would creating an 84 band tiff work? Without seeing the data nor knowing exactly what you'd like to do another option would be to create two empty folders (summer and winter) and then extract the necessary bands (per year) to create new tiffs (one for each season each year). The outputs go into the corresponding folders with a name like "year01_summer". Then you can iterate through each folder applying the necessary formula. It may not be the most efficient approach and think ahead to any other processing you may wish to do. Let me know if that helps and I'd be happy to make a demo model is you can send 3 years worth of data or let me know where I can download similar data.
burdGIS I think I did not explain myself very well. sorry! So I will try my best:
I have a bunch of raster (period 2011-2018) (.tif) that originally come from a netcdf file. This raster set contains information about different variables (temperature, rain, etc). Some of the raster come in a multiband format (12 bands, one per month) or in a single format (one band of a “x” month of the year)
What I need to do is to create an iterative process in which these years are going to be transformed into a summer and winter. So instead of having 12 bands or months per variable per year, I will end up with 2: summer and winter per year. For example
Summer 2011 = Avg(March,April,May,June,July,August) 2011
Winter 2011-2012 = Avg(September,October,November,December)(2011) & Avg(January,February)(2011+1)
Summer 2012 = Avg(March,April,May,June,July,August) 2012
Winter 2012-2013 = Avg(September,October,November,December)(2012) & Avg(January,February)(2012+1)
And so on, until cover the whole period (2011-2018) with 14 rasters
. One site where you can download (but is daily data) is daac.ornl.gov/cgi-bin/dataset_lister.pl?p=32 Thanks again ;)
awwww. i love you maannnnnn.
Haha, thanks Amiro
Thanks, i don't have netcdf_clip toolbox, how can i get?
Hi mohd. Thanks for the question. I'll take a look at this. Can you tell me which license you have?
How can we also iterate and get each time slice as an individual raster?
Thanks Sofia. This has been asked alot and so I'm putting together a video response.
@@burdGIS Good day, I'm also interested in that one. For time series analysis. Anything yet
The Model Builder crashes after iterating through 14 files. I've tried using different processes but I always get the same result.
Thanks wild onion. What version of Arc are you using and can you post the error message? If it crashes after 14 files how many more do you have?
Hi, I followed your steps and I manage to make the model run with my .nc files. The problem is that the model runs all the files with the same name initials, but when it starts to read my other files in the same folder (with different name initials) it stops the process and gives me errors messages. I am really a beginner and have no idea why the model it is not able to iterate through the other files with different names. Could you help?
+Nathan S. Debortoli thanks for the detailed explanation of what is happening. Would you be able to email me a sample of your data so I could take a look? The model in the video is quite specific to the data from Ghana so it would be a fun challenge to write a more robust model.
Yes sure, where can I find your email?
Nathan S. Debortoli info (at} burdgis dot com! :)
how can i get netcdf file ??????
Hi Shaban, you can get netcdf files from lots of different places. I think NASA have some, NOAA certainly will. You can also make your own with ArcGIS if you have suitable data. If you need any further help with netCDFs please let me know.
Here's some oceanic data from NOAA (not looked at the actual data but it's definitely in netCDF format)
www.nodc.noaa.gov/OC5/WOA09/netcdf_data.html
thank you soo much