HarvestingClouds
HarvestingClouds
  • 35
  • 499 821
Getting started with Big Data in Azure - Azure Data Lake Storage (ADLS) and Azure Data Factory (ADF)
Getting started with Big Data in Azure with Azure Data Lake Storage and Azure Data Factory. This session was presented at the Global Azure 2022 event. This session included detailed discussion and demos around getting started with Big Data in Azure. The session focused on Azure Data Lake Storage Gen 2 (ADLS Gen2) and Azure Data Factory (ADF).
The slide deck can be downloaded here: github.com/HarvestingClouds/SessionsAndDecks/tree/main/Global%20Azure%202022
Session content includes:
00:00 Overview and Agenda
00:43 Introduction
01:30 Data Storage Concepts
13:13 Demo - Azure Data Lake Storage (ADLS) - Gen 2
17:39 Azure Data Factory Concepts
26:46 Demo - Azure Data Factory
42:14 Certification Path - Azure Data Engineer Associate
43:06 Getting setup with free Azure account
43:39 Key Takeaways and Next Steps
44:41 References, Q&A and thanks
Переглядів: 1 569

Відео

Building a Strategy for Periodic Optimization of Azure Environments - with Eric Bogenschuetz
Переглядів 1,8 тис.2 роки тому
Building a strategy for periodic optimization of Azure environments with Eric Bogenschuetz. This session was presented at the Global Azure 2022 event. This session includes practical strategies for optimizing 4 key focus areas: 1. Security 2. Cost 3. Performance 4. Supportability/Management The slide deck can be found here: github.com/HarvestingClouds/SessionsAndDecks/tree/main/Global Azure 202...
Connecting Virtual Networks in Microsoft Azure via Peering
Переглядів 37 тис.3 роки тому
In this video, we take a practical look at Virtual Network peering (or commonly referred to as vNet peering). We look at all the caveats and also break down various myths. Virtual Network peering is the way you connect two different Virtual Networks. It is one of the most common things that you will require while setting up networking in any environment. It enables you to seamlessly connect two...
Showdown - Service Endpoints vs Private Endpoints in Microsoft Azure
Переглядів 49 тис.4 роки тому
In this video, we compare Service Endpoints and Private Endpoints in Microsoft Azure. Both the features allow you to easily connect to Azure Services from your Virtual Networks. We look at six different ways these two services are different from each other. We look at the architecture and how each of the services works before jumping into the comparison. Before you watch this video, I highly re...
How to create Service Endpoints for Virtual Networks in Azure
Переглядів 31 тис.4 роки тому
In this video, we explore Service Endpoints in Microsoft Azure. This is a feature of Virtual Networks that provides you better connectivity to various Azure resources. In the video, we look at connecting to Azure Storage Accounts from Virtual Machines from a subnet within a Virtual Network. The concepts are equally applicable to other supported Azure services like Microsoft Azure SQL Databases,...
Locking Azure Resources - Ultimate protection for your resources
Переглядів 2 тис.4 роки тому
In this video, we are checking how you can protect your resources against any unintended deletions or updates. We explore the Azure Resource Manager Locks with practical hands-on examples in this video. We see how to secure our critical resources so that even administrators and owners can't delete or make unnecessary changes to the most important resources within the cloud. We look at best prac...
How to Create Virtual Networks in Microsoft Azure
Переглядів 41 тис.4 роки тому
In this video, we switch gears to practical aspects of networking. We are seeing the details related to creating a new virtual network or vNet in the Microsoft Azure portal. We create one virtual network with 3 subnets. We look at various restrictions when creating virtual networks. If you need to modify anything, then how to do that after the creation of the virtual network is also explored in...
Designing Networks and performing Networking related calculations - with examples
Переглядів 4,5 тис.4 роки тому
This video builds on the previous video where we saw the fundamentals of IP addresses. That video can be found at this link: ua-cam.com/video/bfV6kRi0U98/v-deo.html In this video, we use the knowledge from the previous video and start designing the Networks. These Networks could be Virtual Networks (vNets) in Microsoft Azure or Virtual Private Clouds (VPCs) in Amazon AWS. We look at 3 examples ...
How IP addresses work - Networking Fundamentals
Переглядів 6 тис.4 роки тому
This video is all about building the right foundation for all the upcoming Networking related videos. In this video, we talk about the basics of IP Addresses. This information may seem simple but is very powerful in understanding the complex concepts later on. We will be dissecting the IP addresses and understanding each component. We also learn about Subnet masking and about different classes ...
Creating an Azure Private Endpoint Connection with Azure Storage Accounts
Переглядів 62 тис.4 роки тому
In this video, we are creating an Azure Private Endpoint connection with Azure Storage Account. We begin by discussing the scenario that we are building in this video and discussing what is it that Azure Private Endpoint Connection is providing us in Azure. We will attempt the connection prior to creating the Private Endpoint connection. And then we will set up the Private Endpoint. After that,...
Understanding Private Endpoints - Azure Services Simplified
Переглядів 66 тис.4 роки тому
In this video, we are exploring what are Azure Private Endpoints. We look at the problem first that Microsoft is solving with Azure Private Endpoints. And then we look at how this works to solve that problem. We take Azure Storage Account as an example and look at how you connect to it without Private Endpoint. And then we look at how Private Endpoint works to provide you better and more secure...
The Ultimate guide to working with Azure Storage Explorer
Переглядів 26 тис.4 роки тому
Azure Storage Explorer is a small GUI based utility for working with Azure Storage Accounts. This utility is very versatile. It provides various ways to connect and work with storage accounts. You can customize the experience with lots of settings available in this utility. It uses the AzCopy command-line utility under the hood, to upload and download the data to and from the cloud. In this vid...
Disaster Recovery of Azure Storage Accounts
Переглядів 9 тис.4 роки тому
In this video, we talk about the disaster recovery of the Azure Storage Accounts. We discuss how you can configure the replication strategy of a storage account and how can you make it more resilient. This helps you if there are any service disruptions within the cloud. This provides you with a guarantee that your data will be available in case of any disastrous event. We configure the Globaly ...
Configuring Firewall and Virtual Network access on Azure Storage Accounts
Переглядів 16 тис.4 роки тому
In the previous video, we saw how to securely accessing Azure Storage accounts using Shared Access Signatures. You can check that video here: ua-cam.com/video/pXsVR67gQ7s/v-deo.html In this video, we are taking the security one step further by limiting the access to storage accounts completely by configuring Firewalls on them. We can limit the access to either particular subnets within specific...
How to securely access Azure Storage Accounts - enhancing security
Переглядів 16 тис.4 роки тому
Azure Storage Accounts provide powerful ways to store your data in the cloud. It also means that you need to do everything you can to ensure that these storage accounts are accessed securely. You have various ways to access the storage accounts. You can leverage access keys to access storage accounts as administrators. Or you can use Shared Access Signature to enhance security. In this video, w...
10 Ways to move your data to Azure Storage - Part 2
Переглядів 3,4 тис.4 роки тому
10 Ways to move your data to Azure Storage - Part 2
10 Ways to move your data to Azure Storage - Part 1
Переглядів 8 тис.4 роки тому
10 Ways to move your data to Azure Storage - Part 1
Azure Storage Accounts - How to Create new Storage Accounts
Переглядів 20 тис.4 роки тому
Azure Storage Accounts - How to Create new Storage Accounts
How to work with AWS Auto Scaling Groups and AWS Auto Scaling service
Переглядів 4,2 тис.4 роки тому
How to work with AWS Auto Scaling Groups and AWS Auto Scaling service
How to Create AWS Launch Configurations and Launch Templates
Переглядів 15 тис.4 роки тому
How to Create AWS Launch Configurations and Launch Templates
How to Configure Autoscaling on Microsoft Azure Virtual Machine Scale Sets (VMSS)
Переглядів 20 тис.4 роки тому
How to Configure Autoscaling on Microsoft Azure Virtual Machine Scale Sets (VMSS)
Creating Microsoft Azure Virtual Machine Scale Sets (VMSS)
Переглядів 24 тис.4 роки тому
Creating Microsoft Azure Virtual Machine Scale Sets (VMSS)
How to protect Azure Virtual Machines (VMs) against any disasters
Переглядів 7654 роки тому
How to protect Azure Virtual Machines (VMs) against any disasters
Microsoft Azure VMs vs Amazon AWS EC2 Instances - Part 6 - Monitoring EC2 Instances
Переглядів 1774 роки тому
Microsoft Azure VMs vs Amazon AWS EC2 Instances - Part 6 - Monitoring EC2 Instances
Microsoft Azure VMs vs Amazon AWS EC2 Instances - Part 5 - Monitoring Azure VMs
Переглядів 2624 роки тому
Microsoft Azure VMs vs Amazon AWS EC2 Instances - Part 5 - Monitoring Azure VMs
Microsoft Azure VMs vs Amazon AWS EC2 Instances - Part 4 - Backing up EC2 Instances
Переглядів 1674 роки тому
Microsoft Azure VMs vs Amazon AWS EC2 Instances - Part 4 - Backing up EC2 Instances
Microsoft Azure VMs vs Amazon AWS EC2 Instances - Part 3 - Backing up Azure VMs
Переглядів 4344 роки тому
Microsoft Azure VMs vs Amazon AWS EC2 Instances - Part 3 - Backing up Azure VMs
Microsoft Azure VMs vs Amazon AWS EC2 Instances - Part 2 - Creating EC2 Instances
Переглядів 9304 роки тому
Microsoft Azure VMs vs Amazon AWS EC2 Instances - Part 2 - Creating EC2 Instances
Microsoft Azure VMs vs Amazon AWS EC2 Instances - Part 1 - Creating Azure VMs
Переглядів 3,5 тис.4 роки тому
Microsoft Azure VMs vs Amazon AWS EC2 Instances - Part 1 - Creating Azure VMs
Azure Automation - Hybrid Workers
Переглядів 6 тис.6 років тому
Azure Automation - Hybrid Workers

КОМЕНТАРІ

  • @Momentum_Option_Buyer
    @Momentum_Option_Buyer 3 дні тому

    12:52 Is the VM ending with 1.130 a bastion host within the VNet where subnet of Private Endpoint resides?

  • @ErTarunAgarwal
    @ErTarunAgarwal 13 днів тому

    It would be great if you give better insights on how it appears to be a connection using private ips in case of private endpoint connections.

  • @sanjaygopal627
    @sanjaygopal627 Місяць тому

    Thanks for a great video. Can a service endpoint in Tenant A access resources in Tenant B? Not sure if service endpoints is the approach to take when resources in one tenant need to be made availabe to consumers in another tenant.

  • @vijayalakshmiu4337
    @vijayalakshmiu4337 Місяць тому

    We have few appservices in 2 subnets of single vnet. Now the communication between webapps from subnet 1 to webapps of subnet2 is configured via private end point. But it is not working and giving IP forbidden error. Please suggest somw solutions bro

  • @melsoliano1937
    @melsoliano1937 2 місяці тому

    well explained, keep up the good work mate!

  • @shoumou9504
    @shoumou9504 2 місяці тому

    Thank u.

  • @AkshayGupta108
    @AkshayGupta108 2 місяці тому

    True to title "Simplified.. " Thanks...

  • @juancarlos8552
    @juancarlos8552 3 місяці тому

    Thank you very much for the clarity of your explanation. Could I use this type of connection to connect my power automate with my storage account located in two different regions? What else should I configure apart from what you show in the video? What should I add to the NSG? Thank you for your support.

  • @upulwaruna.abeysinghe
    @upulwaruna.abeysinghe 3 місяці тому

    a best explanation thank you

  • @kasperskyns
    @kasperskyns 3 місяці тому

    Best explanation so far. Good work

  • @hanumantshinde5652
    @hanumantshinde5652 3 місяці тому

    Thanks for the video. I understood GRS completely and I would like to know in a similar way how ZRS and LRS works

  • @gerardbaste5265
    @gerardbaste5265 3 місяці тому

    This is by far the best explanation I have seen on this topic, you did a fantastic job here !

  • @LuisOlias
    @LuisOlias 3 місяці тому

    Thanks for sharing your amazing knowledge! , greetings from Spain.

  • @NareshKumar-qm3em
    @NareshKumar-qm3em 4 місяці тому

    What is case if we have both private end point and service endpoint storage resource

  • @mukeshkane
    @mukeshkane 4 місяці тому

    Great video A help needed, my client is not able to connect to my storage account because the ingress IP for them is coming as a private IP while we have whitelisted their public IP. Any suggestions how can we force their servers to use only public IP. PS: client's servers are also in the azure and possibly same region

  • @shaikmeeravali
    @shaikmeeravali 4 місяці тому

    Excellent explanation

  • @venugopalreddy
    @venugopalreddy 4 місяці тому

    This great. I was trying to do this Service endpoint for Azure DB for PostgreSQL and I am not able to do it. Can you please guide me/Make a video for the same?

  • @jarecheiroso7460
    @jarecheiroso7460 5 місяців тому

    thanks! : )

  • @complexity8851
    @complexity8851 5 місяців тому

    Just had one doubt, if I enable a private endpoint for one of my storage accounts, will it disable all access via public internet?

  • @user-zm4kv4ww9x
    @user-zm4kv4ww9x 5 місяців тому

    Hello @HarvestingClouds sir, Thank you for the video. We have an Azure SQL server and a SQL private endpoint, with no NSG or route table attached. We have already established VPN connectivity between the on-premises server and Azure using Azure site-to-site VPN. We have an Azure firewall and an on-premises firewall. We want to connect from the on-premises server to the Azure SQL private endpoint. Can you please guide us on how to do that? Do we have to open a port in both the Azure firewall and the on-premises firewall, and also add the on-premises firewall public IP addresses to the Azure SQL database firewall configuration in networking? Or is any one option enough?

  • @InstituteForCloudComputing
    @InstituteForCloudComputing 5 місяців тому

    This playlist is just wow. But since we are comparing AWS and Azure, i wish you present how internet gateway, rout table and NAT gateway functionalities are handled in Azure. Also may be a presentation on how any database is used in Azure. May God continue to bless you

  • @rs-tarxvfz
    @rs-tarxvfz 5 місяців тому

    Nice try, but you are just parroting the the things without explaining.

  • @anthonyp3961
    @anthonyp3961 5 місяців тому

    How would you access the storage account using a web browser? This doesn't seem to work?

  • @boseashish
    @boseashish 5 місяців тому

    thanks a lot

  • @shubhamkalra-th4lp
    @shubhamkalra-th4lp 5 місяців тому

    Crisp and Clear 😀

  • @shubhamkalra-th4lp
    @shubhamkalra-th4lp 5 місяців тому

    Crisp and Clear 😎

  • @barwalgayatri4655
    @barwalgayatri4655 5 місяців тому

    Hi when new machine spins in, how ecommerce application will be loaded in it ? It will be a completely new vm right?? I am trying to search this answer but not getting it anywhere

  • @g.lingak4562
    @g.lingak4562 5 місяців тому

    ❤ it, thanks for this video

  • @cyrilfacey8534
    @cyrilfacey8534 6 місяців тому

    One of the best tutorials I've come across. I hope that you will continue to introduce more UA-cam video content.

  • @mihaneman3129
    @mihaneman3129 6 місяців тому

    thank you so much

  • @mihaneman3129
    @mihaneman3129 6 місяців тому

    muchas gracias

  • @helloharshad
    @helloharshad 6 місяців тому

    Wow! I came across this video after 3 Years, and its explained so well and in a very simple way with example. I understood it for good, you presented it so well, thank you.

  • @padmanathanramasamy8354
    @padmanathanramasamy8354 6 місяців тому

    During the failback process, when I change the data replication back to GRS. I understand WestUS will become primary and eastUS will become secondary. How do we change back to original state. I mean Eastus become primary and westus as secondary. Do I need to repeat the same steps?

  • @AllenOlayiwola
    @AllenOlayiwola 6 місяців тому

    Thanks, great video

  • @srilatha3643
    @srilatha3643 6 місяців тому

    videos are really great! please do more videos on AKS

  • @user-em3gw8on5i
    @user-em3gw8on5i 7 місяців тому

    hi, can anybody please help? Is that possible to set the specific secondary region (geo-replication) with own choice? ****Very important****, I need this help on priority, so I will really appreciate it if you could help. TIA

  • @chefe417
    @chefe417 7 місяців тому

    When trying to access the storage account from the VM,... at 2:34 you're saying that it doesnt leave the MS backbone. Also you say it goes over the internet. I am new to networking so maybe I just not firm with definitions, but I would have thought that the MS backone is NOT the internet. And therefore, accessing the storage endpoint over (e.g. a service endpoint) is private. Can you please explain my error in thinking?

  • @allenbythesea
    @allenbythesea 7 місяців тому

    what about point to site vpn?

  • @ramin9842
    @ramin9842 7 місяців тому

    Never came across such a precise, concise and to the point explanation on the topic so far. keep it up

  • @ahzidmahmood6904
    @ahzidmahmood6904 8 місяців тому

    i love you so much, after 3 days of finding an accurate and good example of learning this content today I can that you are the best teacher vs all the cloud gurus have

  • @user-zw6rv6jh5d
    @user-zw6rv6jh5d 8 місяців тому

    Hi HarvestingClould Team, Your classes are really super and, am looking for one particular topic is, How to give the storage account access to external user on particular time for every day

  • @safkaify7875
    @safkaify7875 8 місяців тому

    Nicely explained. Helped me a lot. Please keep up your good work.

  • @many214
    @many214 8 місяців тому

    Thank you for the explanation great video!

  • @Oomtet
    @Oomtet 8 місяців тому

    thank you you make it sound very simple.

  • @SuperAnildesai
    @SuperAnildesai 8 місяців тому

    Hi, instead of using a virtual machine host-level metric, can we use a custom apache2 guest-level metric in VMSS, to scaleup the instance based on request per minute.? any suggestion for this.

  • @mohamedsulthan8027
    @mohamedsulthan8027 9 місяців тому

    How did you created the vm?

  • @mihaneman3129
    @mihaneman3129 9 місяців тому

    thank you so much

  • @stefanberggren770
    @stefanberggren770 9 місяців тому

    At 8:30 he says "you should only have on eprivate endpoint per vnet. Why??

  • @prasenjitgiri919
    @prasenjitgiri919 9 місяців тому

    how come 20 being less didnt fall in the bucket 24 being more than 22?? that explanation made no sense

    • @XtianApi
      @XtianApi 9 місяців тому

      The higher the bit, 20, 22, 24, etc, the fewer the amount of addresses it is referring to. For example, if you are putting a single ip address in somewhere and only want to represent that one single IP address, and nothing else, then it would be /32. If, for example, you have an address space that ends in a very low single digit number, then you are talking about a large portion of all the addresses in existence. Here is a real life idea that may help. If I give you a whole pizza, 1, then I am giving you all of it. But, if I give you only a tiny portion of the pizza, for example 1/24th of the pizza (/24) then I am only giving you a very small portion of the whole pizza. I hope that helps. You can watch a UA-cam video on how all the math works. Use online subnet calculators when you need to.

  • @user-fk9zr5mj7e
    @user-fk9zr5mj7e 10 місяців тому

    Thanks such a great video. I follow all the instructions and it works.