- 143
- 28 508
Tim Higgins
United States
Приєднався 9 січ 2017
I am a Senior Enterprise Technical Architect at A&I Solutions, Inc. I have created a set of videos to share that I thought was useful for my team and others.
My passion is music and building software... expect some music videos in the future ....
My passion is music and building software... expect some music videos in the future ....
Medallion Architecture - Video 13 -Talk about the next steps for Gold by using Aws ETL to move data
Building out a Medallion Architecture from scratch with Python, Aws Boto3, Aws S3, Aws Glue and Aws Athena
Video 13 Talk about the next steps for Gold by using Aws ETL to move data
I had a use case for handling a large amount of data for querying. I used the Medallion Architecture design to build this out. I thought it would be fun to share this with you all in a real-world working session... Lots of videos but it really didn’t take long, concerning we started from scratch.
dataengineering.wiki/Concepts/Medallion+Architecture
Video 13 Talk about the next steps for Gold by using Aws ETL to move data
I had a use case for handling a large amount of data for querying. I used the Medallion Architecture design to build this out. I thought it would be fun to share this with you all in a real-world working session... Lots of videos but it really didn’t take long, concerning we started from scratch.
dataengineering.wiki/Concepts/Medallion+Architecture
Переглядів: 6
Відео
Medallion Architecture - Video 12 Gold Copy - Aws Crawler/ Aws Athena / Aws ETL / Aws QuickSight
Переглядів 1519 годин тому
Building out a Medallion Architecture from scratch with Python, Aws Boto3, Aws S3, Aws Glue and Aws Athena Video 12 Look at our data in Athena/ looking at an Aws ETL tool and introducing Aws Quicksight for dashboards I had a use case for handling a large amount of data for querying. I used the Medallion Architecture design to build this out. I thought it would be fun to share this with you all ...
Medallion Architecture - Video 11 Look at and update our crawler/ Introducing Aws Athena
Переглядів 319 годин тому
Building out a Medallion Architecture from scratch with Python, Aws Boto3, Aws S3, Aws Glue and Aws Athena Video 11 Look at and update our crawler/ Introducing Aws Athena I had a use case for handling a large amount of data for querying. I used the Medallion Architecture design to build this out. I thought it would be fun to share this with you all in a real-world working session... Lots of vid...
Medallion Architecture - Video 10 Setting up our Aws Glue Crawler and using the Aws Data Catalog
Переглядів 1119 годин тому
Building out a Medallion Architecture from scratch with Python, Aws Boto3, Aws S3, Aws Glue and Aws Athena Video 10. Setting up our Aws Glue Crawler and using the Aws Data Catalog I had a use case for handling a large amount of data for querying. I used the Medallion Architecture design to build this out. I thought it would be fun to share this with you all in a real-world working session... Lo...
Medallion Architecture - Video 9 finishing our ETL/Silver by getting our data ready for Aws Glue
Переглядів 1319 годин тому
Building out a Medallion Architecture from scratch with Python, Aws Boto3, Aws S3, Aws Glue and Aws Athena Video 9. Finishing our ETL to Silver layer by getting our data ready for Aws Glue I had a use case for handling a large amount of data for querying. I used the Medallion Architecture design to build this out. I thought it would be fun to share this with you all in a real-world working sess...
Medallion Architecture - video 8 Continuing our ETL to Silver by building the Merge layer to Aws S3
Переглядів 1619 годин тому
Building out a Medallion Architecture from scratch with Python, Aws Boto3, Aws S3, Aws Glue and Aws Athena Video 8. Continuing our ETL to Silver by building the Merge layer with Python Pandas to Aws S3 Buckets I had a use case for handling a large amount of data for querying. I used the Medallion Architecture design to build this out. I thought it would be fun to share this with you all in a re...
Medallion Architecture - Video 7 Starting the ETL to Silver
Переглядів 3319 годин тому
Building out a Medallion Architecture from scratch with Python, Aws Boto3, Aws S3, Aws Glue and Aws Athena 7. Starting our Silver Layer by building an ETL with Python, Aws Boto3, and Aws S3 Buckets I had a use case for handling a large amount of data for querying. I used the Medallion Architecture design to build this out. I thought it would be fun to share this with you all in a real-world wor...
Medallion Architecture - Video 4 Building our Bronze layer by starting our code in Python and Boto3
Переглядів 6319 годин тому
Building out a Medallion Architecture from scratch with Python, Aws Boto3, Aws S3, Aws Glue and Aws Athena Video 4. Building our Bronze layer by starting our code in Python and Boto3 I had a use case for handling a large amount of data for querying. I used the Medallion Architecture design to build this out. I thought it would be fun to share this with you all in a real-world working session......
Medallion Architecture - Video 5 Sending a bulk amount of data to Aws S3 using Python/Boto3
Переглядів 5419 годин тому
Building out a Medallion Architecture from scratch with Python, Aws Boto3, Aws S3, Aws Glue and Aws Athena Video 5. Coding our Bronze layer by continuing with Bulk upload to Aws s3 with Python and Boto3 I had a use case for handling a large amount of data for querying. I used the Medallion Architecture design to build this out. I thought it would be fun to share this with you all in a real-worl...
Medallion Architecture - Video 6 Coding our Bronze layer by refracting (cleaning) our code
Переглядів 1219 годин тому
Building out a Medallion Architecture from scratch with Python, Aws Boto3, Aws S3, Aws Glue and Aws Athena Video 6. Coding our Bronze layer by refracting (cleaning) our code I had a use case for handling a large amount of data for querying. I used the Medallion Architecture design to build this out. I thought it would be fun to share this with you all in a real-world working session... Lots of ...
Medallion Architecture - Video 3 Start our Bronze layer by looking at our data
Переглядів 4919 годин тому
Building out a Medallion Architecture from scratch with Python, Aws Boto3, Aws S3, Aws Glue and Aws Athena 3. Start our Bronze layer by looking at our data I had a use case for handling a large amount of data for querying. I used the Medallion Architecture design to build this out. I thought it would be fun to share this with you all in a real-world working session... Lots of videos but it real...
Medallion Architecture - Video 2 Create a bulk amount of realistic data with Broadcom TDM for our LZ
Переглядів 6319 годин тому
Building out a Medallion Architecture from scratch with Python, Aws Boto3, Aws S3, Aws Glue and Aws Athena 2. Create a build amount of realistic data with Broadcom TDM for our LZ I had a use case for handling a large amount of data for querying. I used the Medallion Architecture design to build this out. I thought it would be fun to share this with you all in a real-world working session... Lot...
Medallion Architecture - Video 1 Building out a Medallion Architecture from scratch w/ Python & Aws
Переглядів 6219 годин тому
Building out a Medallion Architecture from scratch with Python, Aws Boto3, Aws S3, Aws Glue and Aws Athena. Video 1. What is the Medallion Architecture design and how should we get started? I had a use case for handling a large amount of data for querying. I used the Medallion Architecture design to build this out. I thought it would be fun to share this with you all in a real-world working ses...
Using Broadcom OI APIs to Export Metrics
Переглядів 53День тому
This is for learning the process, I don't actually recommend this because OI has the power to show you everything and then some. knowledge.broadcom.com/external/article/206461/aiops-how-to-query-for-metrics-and-top.html#mcetoc_1fic5bbldv
aws S3 buckets from the cli (command line)
Переглядів 55День тому
aws S3 buckets from the cli (command line)
A quick video for setting up JetBrains with Aws and run the LLM demos locally
Переглядів 75Місяць тому
A quick video for setting up JetBrains with Aws and run the LLM demos locally
Aws Serverless architect with Aws API Gateway, Foundation Models, Lambda and S3 buckets series
Переглядів 36Місяць тому
Aws Serverless architect with Aws API Gateway, Foundation Models, Lambda and S3 buckets series
Aws Serverless architect with Aws API Gateway, Foundation Models, Lambda and S3 buckets series
Переглядів 72Місяць тому
Aws Serverless architect with Aws API Gateway, Foundation Models, Lambda and S3 buckets series
Aws Serverless architect with Aws API Gateway, Foundation Models, Lambda and S3 buckets series
Переглядів 32Місяць тому
Aws Serverless architect with Aws API Gateway, Foundation Models, Lambda and S3 buckets series
Aws Serverless architect with Aws API Gateway, Foundation Models, Lambda and S3 buckets series
Переглядів 22Місяць тому
Aws Serverless architect with Aws API Gateway, Foundation Models, Lambda and S3 buckets series
Aws Serverless architect with Aws API Gateway, Foundation Models, Lambda and S3 buckets series
Переглядів 19Місяць тому
Aws Serverless architect with Aws API Gateway, Foundation Models, Lambda and S3 buckets series
Aws Serverless architect with Aws API Gateway, Foundation Models, Lambda and S3 buckets series
Переглядів 36Місяць тому
Aws Serverless architect with Aws API Gateway, Foundation Models, Lambda and S3 buckets series
Get data from Broadcom Spectrum and PM/DA with Python into Json/CSV files part 9-9 Final Thoughts
Переглядів 132 місяці тому
Get data from Broadcom Spectrum and PM/DA with Python into Json/CSV files part 9-9 Final Thoughts
Get data from Broadcom Spectrum and PM/DA with Python into Json/CSV files part 8-9 deploying
Переглядів 132 місяці тому
Get data from Broadcom Spectrum and PM/DA with Python into Json/CSV files part 8-9 deploying
Get data from Broadcom Spectrum and PM/DA with Python into Json/CSV files part 6-9
Переглядів 102 місяці тому
Get data from Broadcom Spectrum and PM/DA with Python into Json/CSV files part 6-9
Get data from Broadcom Spectrum and PM/DA with Python into Json/CSV files part 5-9
Переглядів 102 місяці тому
Get data from Broadcom Spectrum and PM/DA with Python into Json/CSV files part 5-9
Get data from Broadcom Spectrum and PM/DA with Python into Json/CSV files part 7-9
Переглядів 62 місяці тому
Get data from Broadcom Spectrum and PM/DA with Python into Json/CSV files part 7-9
Get data from Broadcom Spectrum and PM/DA with Python into Json/CSV files part 2-9
Переглядів 182 місяці тому
Get data from Broadcom Spectrum and PM/DA with Python into Json/CSV files part 2-9
Get data from Broadcom Spectrum and PM/DA with Python into Json/CSV files part 4-9
Переглядів 82 місяці тому
Get data from Broadcom Spectrum and PM/DA with Python into Json/CSV files part 4-9
Get data from Broadcom Spectrum and PM/DA with Python into Json/CSV files part 3-9
Переглядів 202 місяці тому
Get data from Broadcom Spectrum and PM/DA with Python into Json/CSV files part 3-9
You didn't get a single like in 4 years, but your one point in this whole video helped me a lot, thanks.
thank you very much!!! I am glad it helped!
Hi Tim, its a great video, but I would like to know more about Pre-Scripts & Post Response Scripting. I tried with Jscript, but looks its still failing with attributes. If you could help me with your vide if you made it already or any document where i can refer them. It would be a great help & thanks in advance.
Hi Tim, Have a question. How do we create relationships between 2 or more tables in the process of synthetic data creation.Thanks in advance
Hi, I can show you that but give me a few days. If you are doing Data Gen, in the middle of the data painter you can choose columns form other tables. They are other ways but this is a quick answer. Let me know if this helps or I can explain it more if needed.
Thanks for this wonderful video, it really helped, I have used Postman API testing tool extensively, this just looks so similar to me.
Tim, Great stuff. Loving your audio quality and the interviews... A whole other world out there beyond our own circles... Thanks for sharing!!
Thanks for this great opportunity to talk tech, gear and history Tim! I really enjoyed it and thanks for coming up with the cool questions! ;-)
Thanks man... the honor was all mine. I truly enjoyed it and we must do it again, soon.
Thanks Tim! How awesome. Thanks Joe. Hey Tim, great "studio" set up. Wokld live to know more about your recording process... Id like to improve my audio and video from my "almost studio"... Ok, not studio... My man room. Thanks!
yea man... 1st thank you!!! 2nd, lets get together and I will be glad to show you!!!
is this tool opensource or Licensed ?
Sorry its Licensed through Broadcom but a GREAT TOOL
is this tool opensource or Licensed ?
Do you have any plan to post a video of DevTest being deployed on OpenShift?
let me see if i can get one together... if not i'll get you to the right place for instructions
ي❤❤»❤
Muy buen video
Sweet!
How can eastern countries get you
can you send me a private message. I am not sure if youtube will do that
I need also to join you because I have liked your business
Sweet !
At what point of time we create Travel_E schema? Is this one of the pre-requisites before we could start creating even the Project- Dataset - Pool in Data Maker? This means we require DBA's support and expose PII data before we could do subsetting and on the fly masking? Great Video by the way!
Yes when you download TDM one of the options is to download the demo databases. Sadly TDM is a database tool so you will need to interact with the database sometimes... but you could also use files if you are just learning. You can do fly masking with Subsetting, I have a video on masking from subsetting. see if you can find it.
Thank you for explaining each and every concept in the portal. I have one doubt.. how to run pre or post SQL in the portal data masking
Hello Sorry for the late response but you can run a Javelin in your publish in a pre or post publish Look at the Javelin video
Thanks for sharing your knowledge. helped me a lot in my company, hope you put more videos thanks
thank you Daniel. I will put more videos up soon.. Im glad to hear this helps.
I have 2.5 years experience in devtest and now I'm searching for job could you refer me
Great stuff Tim. Thanks for the video!
Thanks man!
Who knew ??... Awesome video. Thanks Tim!
Short vids help close the loops! Thanks Tim!
Now the advanced stuff! Digging deep now! Keep the videos coming
thanks man... I thought of you while I was doing this :)
Such a great video.
short and sweet.. I try... thank you!
How can i add data rules on my table column in generator?
can you give me an example of what you are looking for? If you are talking about if user = tim then do this... then you can use the data painter for that. I have some videos on that.
@@timhiggins4440 yes exactly.my requirement is i have a table with no data.i need to create some test datas for that table and export to excel.but i need to do some data rules like for the coloumn SSN,FName,last name.the challenge is i need to take some value in a seed table and need to use that in rules.. Could you give your vedio link for data rules
@@timhiggins4440 exmple validation for SSN columns
Nobody gets it done like Tim. Great stuff!
Now we are getting to the good stuff!
well thank you sir!!!! we need to have game night soon!!
Ahhh 64 bit!........ Now it all makes sense. Tim, you're the man!
Glade I can help you out my friend.... ha ha.. you can always count on me ...
Nice!
Awesome follow up video on part 1!....
ha ha.. thanks man
Finally! Great info!
Hi Tim, Can you create a video on how to configure GT maker with my tdm repository
Yes I will be glad to, I am making a GT Datamaker set of videos today. Tell me if this helps, or I can give more info.
see if this helps ua-cam.com/video/g67UqyvBFyU/v-deo.html
Sorry I had to reupload the video... try this one ua-cam.com/video/-QOvNvdlehM/v-deo.html
@@timhiggins4440 unfortunately I delete the gtrep connection in GTmaker. Let me try to connect with 64bits odbc with mentioned configuration. I appreciate your effort on help make configuration. I'll update you the result shortly
@@ark5048 good luck.. let me know if you get it working
I bet you can do a great Eddie Van Halen impersonation. You sound just like him.
I'll take that.. I wished..
@Tim Higgins Hello, Can you please provide your emai id ? I want to reach out to you ?
Thank goodness! .... Soooo been waiting for this!.
Ha ha.. I've missed you man... how are you?
Hi Tim.. Can you help me that i need to get trained in ARD
Sadly I am not sure, Broadcom did have some sites for training. My videos should help a lot. If you need more, I would try Broadcom.
Hey Tim, great video .. thanks so much! appreciate it ..
Can we re run the masking job if the test DB is refreshed with the production data ? It happens in regular interval in my organization
you can save your masking and run it when ever you need it. That way if production data comes down, you can re run the masking
Finally! We've been waiting on this one!
I was hoping you would need this one.
Great stuff!
thanks man!!!
Very good video. But a humble request can you please show like how can we take api key, entity from newrelic to integrate blazmeter to it. I tried it but am not able to get entity in blazmeter.
sorry I have been busy but yes I will be glad to. are you still having issues with this
@@timhiggins4440 It will be good for all of us if you add that video aswell
That filled in the blanks!
Nice!!
Finally! The one I was waiting for!.. Great Information Tim.
thanks a lot man
Appreciate your efforts Tim. Is there anyway I can learn all the components of TDM ? Complete course or something ?
sadly Broadcom has training but its not free. If you watch all my TDM videos I cover them all. My company can also provide training.
Hi
Can I know how to enroll for training ?
Is there any trial version available?please give the link
Hey man, sorry I am the architect and I don't do sales. If your company is looking to try TDM, email me and i can get you in touch with my company.
Probably the best 22 minute instructional video I've seen today! Great stuff. Keep it coming. Thumbs up from me!
thanks brother..
Hi Tim, Are you with Broadcom? we are looking for corporate license. Do you have any clue how to get it?
Sorry for the late response. We are a partner with Broadcom; you company should have a sales rep. If this isnt to late send me a private message with your email and I can get someone to help you.
Hi Tim, how are you doing? suppose you have a use case where we need to pull and display columns from more than one table after joining the tables using relationship, how would you do that in Find and reserve? I tried but I am not able to design Find and reserve request such that columns across 2 to 3 tables are populated in the results table after you click on Find Data from self service.
When doing Find and Reserve, you want to create your model 1st. You can create relationships in the model that you want to use. Try that... that should work for you. Let me know if that helps but spend more time creating your relationships with the data model
Thanks Tim. Thank you so much for the detailed Demo
Thank you very much. Please let me know if you have any questions.