- 20
- 64 167
Aman Bagrecha
Приєднався 30 лис 2013
Daily learnings turn into Videos and Blogs. Find me on amanbagrecha.github.io/
Understanding Jason-3 Satellite Products: Data and Download
This video explains various Jason3 satellite products and how to download them from various portals using both manual and programmatic way to download the data.
Links
blog post - amanbagrecha.github.io/post/rs_gis/understanding-jason-3-satellite-data-and-its-applications/
github code - github.com/amanbagrecha/jason3/tree/main
podaac nasa - podaac.jpl.nasa.gov/dataset/JASON_3_L2_OST_OGDR_GPS
Jason-3 product handbook - www.ospo.noaa.gov/Products/documents/hdbk_j3.pdf
Mostly active on twitter aman_bagrecha
else amanbagrecha.github.io/
#jason3
#remotesensing
#noaa
Links
blog post - amanbagrecha.github.io/post/rs_gis/understanding-jason-3-satellite-data-and-its-applications/
github code - github.com/amanbagrecha/jason3/tree/main
podaac nasa - podaac.jpl.nasa.gov/dataset/JASON_3_L2_OST_OGDR_GPS
Jason-3 product handbook - www.ospo.noaa.gov/Products/documents/hdbk_j3.pdf
Mostly active on twitter aman_bagrecha
else amanbagrecha.github.io/
#jason3
#remotesensing
#noaa
Переглядів: 1 105
Відео
Import CSV and OSM data into PostgreSQL using ogr2ogr
Переглядів 5972 роки тому
In this video we use ogr2ogr to insert CSV and OSM data into PostgreSQL. We install postgis and hstore extension and look at various flags associated with ogr2ogr command. Code: gist.github.com/amanbagrecha/4a3394a5153241a45877b3779866c663 Data: 1. bahamas-latest.osm.pbf download.geofabrik.de/central-america/bahamas-latest.osm.pbf 2. filter_all_cat_data.csv github.com/amanbagrecha/amanbagrecha....
Download MODIS data using Python
Переглядів 4,4 тис.2 роки тому
Using modis-tools python library, we can easily download modis dataset. The python package uses CMR API to search for granules. Blog: amanbagrecha.github.io/post/rs_gis/download-modis-data-using-cmr-api-in-python/ slides: docs.google.com/presentation/d/1Zk6LrpfGNlveIZ7RhQ6urtlXsLoKjhSWOphnEEQUEqU/edit#slide=id.gc6f75fceb_0_0 NASA CMR API: cmr.earthdata.nasa.gov/ Python package: github.com/fraym...
Train Spacenet 5 Road Extraction Algorithm on AWS Spot Instance
Переглядів 1282 роки тому
Extract road network using CRESI baseline algorithm using modified docker image. We train on 30cm image of 512x512 pixel size and get the road network from the image. Project repo: github.com/amanbagrecha/cresi-modified Mostly active on twitter aman_bagrecha else amanbagrecha.github.io/ #Spacenet #DeepLearning
Spacenet 5 CRESI Road Extraction
Переглядів 5102 роки тому
Extract road network using CRESI baseline algorithm using modified docker image. We test on 60cm image of 512x512 pixel size and get the road network from the image. Project repo: github.com/amanbagrecha/cresi-modified Mostly active on twitter aman_bagrecha else amanbagrecha.github.io/ #Spacenet #DeepLearning
[Part 2] Making our PostgreSQL database to allow remote connection in Open Data Cube on AWS EC2
Переглядів 7802 роки тому
In this video I show how to configure Open Data Cube and PostgreSQL to access the database from client application such as pgAdmin. [Part 1] ua-cam.com/video/UnPC-BcVzoo/v-deo.html We open the port for postgres and configure 2 files which are detailed here www.postgresql.org/docs/current/auth-pg-hba-conf.html and stackoverflow.com/questions/3602450/where-does-postgresql-store-configuration-conf...
[Part 1] Install Open Data Cube on AWS EC2 and Index Sentinel-l2a-cogs
Переглядів 1,5 тис.2 роки тому
In this video I show how to setup/ Install Open Data Cube and index sentinel-l2a-cogs on a EC2 instance. We also look into STAC API and how it is utilized in indexing the data. Slides: docs.google.com/presentation/d/1wCvt8bqv6afdRxOfKmuNfQ5J7XahWwLznKz14hTcuPM/edit?usp=sharing About STAC: stacindex.org/ About Open Data Cube: www.opendatacube.org/ Download the config file for adding product: git...
Download GPM IMERG 30-min Rainfall Data in netCDF format using Python (part 2)
Переглядів 6 тис.2 роки тому
How to download and process 30-min GPM Rainfall Data Watch the part 1 to understand GPM IMERG data here ua-cam.com/video/PeSXXftKJzs/v-deo.html&ab_channel=AmanBagrecha Notebook for the code to run colab.research.google.com/drive/1VIKun8K3RT8VvcPJ7DE5uDDC10i10k1T?usp=sharing Contact me on twitter aman_bagrecha or any other social media amanbagrecha.github.io/
GeoServer Tutorial: Limit Layer Access
Переглядів 2,5 тис.3 роки тому
We look at how to limit access to specific workspaces in GeoServer by creating Roles and Users. Understand the concept of Data Security and how to assign permissions to Roles, and attach Roles to Users. Read more about layer security at docs.geoserver.org/stable/en/user/security/layer.html Find me on twitter aman_bagrecha
Move, Rotate and style Label in QGIS
Переглядів 3,9 тис.3 роки тому
Positioning lable was hard and tedious in previous QGIS version. With the advent of auxilary-storage, things have become much simpler to begin with. Here we fiddle around with label properties and also maybe find a bug!! Additional read: oslandia.com/en/2017/10/17/auxiliary-storage-support-in-qgis-3/ twitter handle: aman_bagrecha
Generate primary key for layer in QGIS
Переглядів 2,9 тис.3 роки тому
We use the power the QGIS expression and use $id which returns the row number.
Style Geoserver Point Layer with SVG
Переглядів 4 тис.3 роки тому
Upload an SLD file having SVG graphic exported from QGIS and imported to Geoserver. Contact me on twitter aman_bagrecha or any other social media amanbagrecha.github.io/
Download GPM IMERG 30-min Rainfall Data with ASCII format ( Part 1)
Переглядів 4,3 тис.3 роки тому
How to download and process 30-min GPM Rainfall Data data source: disc.gsfc.nasa.gov/datasets/GPM_3IMERGHH_06/summary notebook: colab.research.google.com/drive/1GiROw6A7dGyCQTVE5AqV2ighYIiOi_H2?usp=sharing Part 2: ua-cam.com/video/T_Us4hJxSeI/v-deo.html Contact me on twitter aman_bagrecha or any other social media amanbagrecha.github.io/
Import Shapefile to Postgresql ( via QGIS and Postgis Shapefile Import/Export Manager)
Переглядів 12 тис.3 роки тому
How to import shapefile via QGIS and postgis shapefile import/export manager. Contact me on twitter aman_bagrecha or any other social media amanbagrecha.github.io/
Prestressed Concrete Circular Storage Tank
Переглядів 1,1 тис.3 роки тому
Self-study for the subject Prestressed Concrete references for further research Nawy, E. G., Fundamentals of High-Performance Concrete, 2nd ed., John Wiley and Sons, New York, NY, 2001, 454 pp. Nawy, E.G., Concrete Construction Engineering Handbook, 2nd ed., CRC press, Boca Raton, FL, 2008, 1560 pp. Tadros, M.K. “Expedient Service Load Analysis of Cracked Prestressed Concrete Sections.” Journal...
How to calculate Moment of Inertia in AutoCad
Переглядів 16 тис.5 років тому
How to calculate Moment of Inertia in AutoCad
How to import contacts in Gmail from excel file
Переглядів 925 років тому
How to import contacts in Gmail from excel file
great info, thanks
How to additional download data in xml format. Can you please help me asap.
Hi I am trying to solve a 'missing primary ID' warning that I get when attempting to upload my project to QFieldCloud. I have followed the step in your video to create an ID column however my layer still remains 'read-only' in QField. Do I need to somehow specifiy that this new ID column is the primary key? If so, do you know how?
thank you so much. I spent like 2 days trying to figure this out
How to plot MODIS terra LST data over a specific area using heatmap?
i have a question your mail please !
When I pull your cresi-modified folder there is no test imagery in the folder. Why is that ? Is there an easy fix? I am quite a beginner
I am getting key exception error 'feed' in the resp = self.no_auth.get(params=params, auth=None) feed = resp.json()["feed"] It is in resources line 117
how can i read those files? or can i convert hdf to tiff?
Hello thanks for the tutorial can you make the same one to download from Aviso portal?? Please
can you please try it once, its not working now. can you give me suggestion for another way to download v07
When downloading the nefCDF files, the extensions are nc4. In the video, you are working with nc extension. Can you please update the procedure to handle this format? Thanks in advance. I tried your code by changing the nc to nc4 but it didn't work.
Is it open-access data or subscription-based? If subscription-based, what would be the const?
Hi, thank you very much for both videos part1 and part2. They really helped me to understand ODC and STAC. It would be great if you have now a part 3 where you show, how I really can do requests / queries on the datacube. Do you plan to prepare such a hands-on video?
Hi Aman, while importing the info from pbf file i am getting the WKB geometry column as binary, can you guide in terms of how can this be used as as geom
Very well made video. to the point and clear. I have a question too bro. can i export data from geopackage as well with this method?
Hi, can you please guide me about how to mosaic modis tiles by using hdf-eos utility.
so helpful, thank you for your video.
how to get the importer manager?
How to know about the tiles?
I only want 1 day's data what should I enter in the time range? For ex, I want 21st april's data starting from 0 UTC upto next 24 hours what should I enter in time range then? should it 2015-04-21 to 2015-04-21 or 2015-04-21 to 2015-04-22
Hi Can we run this on spyder/jupyter notebook?
Thanks Aman, this video was very heplful.
Hey! what if we have to download more than 5 years of modis data? we have to make separate files year wise directory for data being downloaded or one directory would be enough if we re run the command will the modis files overwrite? please its urgent do let me know
You could make a single directory - and later split up based on your requirements. As for the overwriting part, yes - this package overwrites the files. You have to manually alter the code to make that check to avoid redownloading the file
@@amanbagrecha can you please guide me about downloading Modis files by their names and how can we check that if our data is 100% downloaded and there's no repetition of files. I'm downloading my data and the directory is Shared folder and within the shared folder i also made year wise folders. Im using that folder as directory while downloading then i copy and paste the files to shared folder when the data is downloaded 100%
Excellent content -- thanks for sharing -- Best Regards .
Thank you so much brother. It helped me.
amazing videos! clear , simple and well presented. Thanks a lot
thank you!
Hi! I tried to upload a sld file created in qgis to geoserver and it says that there’s no validation errors and I submitted it. But egen i try to preview the layer nothing is there( i have connected the style to the layer). The wrong message by pressing f12 is: ”devtools failed to load source map: could not load content from chrome-extension://gighmmpiobklfepjocnamgkkbiglidom/browser-polyfill.js.map: system error: : ERR_FILE_NOT_FOUND ” do you perhaps have a clue about what could be wrong?
no
Thank you! It would be awesome to see how to do a getcapabilities with the credentials from outside
very nice way to download!!!
thanks a lot
i'm trying to run the line xr_xr_df=xr.open_mfdataset('test*.nc') but the following error message appear: ValueError Traceback (most recent call last) <ipython-input-85-4e7c711768ee> in <module> ----> 1 xr_df=xr.open_mfdataset('test*.nc') 3 frames /usr/local/lib/python3.7/dist-packages/xarray/backends/plugins.py in guess_engine(store_spec) 153 ) 154 --> 155 raise ValueError(error_msg) 156 157 ValueError: did not find a match in any of xarray's currently installed IO backends ['netcdf4', 'scipy']. Consider explicitly selecting one of the installed engines via the ``engine`` parameter, or installing additional IO dependencies, see: xarray.pydata.org/en/stable/getting-started-guide/installing.html xarray.pydata.org/en/stable/user-guide/io.html I installed all the packages but still not working.....
I am also getting the same error. Please help !
@sharibislam293 Hi both, had the same issue - it's because the data you are trying to download doesn't exist/isn't on the nasa servers any more. Double check that you are downloading the latest dataset version! The .txt file will be a bit bit!
i run all the code, but i cant find the .csv file. where can I find it?
Do check out the second part. Maybe that will help. As a side note, did you register to earthdata account today? If so, you need to wait for 1 day to gain access to the API
@@amanbagrecha I looked the second part. Appear another error message "did not find a match in any of xarray's currently installed IO backends ['netcdf4', 'scipy']. Consider explicitly selecting one of the installed engines via the ``engine`` parameter, or installing additional IO dependencies, see: xarray.pydata.org/en/stable/getting-started-guide/installing.html xarray.pydata.org/en/stable/user-guide/io.html" but i registered to earthdata last nigh, so I'll wait to complete 1 day to try again, thanks from Brazil for your videos, really useful!
can we flip labels sir? just like mirror the label
Hi Sir! Now, how can someone access the secured layers in leaflet or openlayers with his credentials?
you can refer this code for accessing secured layer - github.com/amanbagrecha/openlayers-geoserver-query/blob/main/gisapp/static/gisapp/js/map.js#L65
@@amanbagrecha Thanks very much!
Thanks for your tutorial. @@amanbagrecha I want to use it for leaflet js.
Good package but it adds additional path to the download path that the download url doesn't exist. I tried to download MOD11A2. However, it raise no file location found.
I tried on my machine as well. Turns out, this is a credentials error. For now urs.earthdata.nasa.gov is unable to process it. Will update if it works
Note: You need to have Python 3.8 and above to work with this package because it using the Typing module. If you have any query on how to download specific data, please feel free to reach out.
Hi Aman, I am getting errors while running download_modis.py file.. And I am unable to debug it. Can you please help?
@@wanderlust_womaniya what is the error you are getting?
@@amanbagrecha I am trying to run the code-lines into anaconda's CMD after creating a new environment. following is the error I am getting "[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain"
@@wanderlust_womaniya Looks like you need to run this command `set SSL_NO_VERIFY=1`. If your company or org has proxy in between, that is causing this error
@@amanbagrecha This shall be added in downloa_modis.py file or shall run it in the cmd directly (I tried latter, doesnt seem to work). Is there anyway to connect with you directly? I feel like I am missing some command here or there.
It's work !! Thanks Man. You're Awsome!!
after i run the last code, i got "requests.get() returned an error code 200" for all url list instead of .csv file. .i've been stuck here for 3 days without any clue. Please help me, Sir ;(
and "requests.get() returned an error code 503" . .What should i do
Error code of 200 is not an error code. It means the request was successful
@@amanbagrecha but my .csv file is empty, Sir.
.csv file is created, but the file is empty, only " " in there
@@teknomodd Try downloading one file manually and see if you get the result. Also 503 is server error, no fault of yours
If you get error 401, that means the program could not find the .netrc file or the credentials are incorrect. For windows OS, place the .netrc file inside C:\Users\"your_username". Make sure after you register for earthdata account to wait for at least a day if there is an error code starting with of 403
Hi Sir , I need some help regarding how to plot this data on Indian map
i have placed .netrc file inside C:\Users\"your_username".But i am getting error 401
Please send me pert one
nice
Wrong information 1 not yet subtract object 2 centroid must be X0,000 and Y0‚0000
Hi !, thank you so much for this tutorial, maybe for GSMap precipitation satellite products is there some ftp download also? Regards Aman
GSmap has different server than imerg, although both being GPM product. There is infact ftp server for GSmap
I'm logged in but it won't let me download. Requests.get -> response me error 401. I tried to create the <.netrc> file but it didn't work. Maybe you know how this work well... thanks
You need to wait for 1 day once you create earthdata account. 401 is unauthorized, so your accounts needs to be activated.
im facing same problem here
The problem is with credentials only. Please check your .netrc file is placed correctly and u have typed correctly inside the file
@@amanbagrecha im using Visual Studio Code, where should i place my .netrc file? , its been 2 days im stuck here. . ;(
If you are using windows, place it under C:\Users\"your_username". It does not matter which IDE you use, only the OS matters. Apologies for not showing how to do this steps in windows. Will add a note in description
Thanks for this informative and helpful video. what if i want to download the bulk variable like you just did but lets say date range 2000 to 2021, is there a line that will be required to be added so that all the files can be downloaded in one folder?
Thanks for the feedback. No, you do not need to change anything as far as code is concerned. But make sure to perform your data download in local machine, else it would time out in google colab.
@@amanbagrecha Thank you. so when you have that data in that date range what coding line can you use to only select one year for example to plot lets say a spatial map of a particular year and not the entire period at once...
@@tumelomohomi2980 you would use the `slice` operation to perform the action. You can look at that keyword in xarray docs, you'd find tons of information
@@amanbagrecha Thank you, much appreciated.
Wonderful insights Aman Sir ! Thank you ❤️
Welcome Akshath! A compliment from you made my day. Deeply appreciate it 🙏
When i moved one label the other label disappeared from the map
Thank you for this explanation. I needed to update our icons from PNG to SVG to allow for greater manipulation (e.g. color).
Welcome 😀. SVG over PNG everytime!