Shipyard
Shipyard
  • 159
  • 95 127
Hakkōda and Shipyard Partnership: What It Means For You
Exciting news! We are thrilled to announce our partnership with the Hakkoda team, the Snowflake experts.
This video discusses how this collaboration will revolutionize data operations and make it easier than ever to create a data-drive culture.
Visit our blog to learn more about this game-changing partnership
#snowflakes #dataoperations #dataorchestration #datamanagement #data #datanengineering
Blog announcement:
Website: Shipyardapp.com
Hakkoda's Website:
Get started with Shipyard (it's free!): app.shipyardapp.com/auth/signup
Follow us on LinkedIn: www.linkedin.com/company/shipyard/
Переглядів: 39

Відео

How to Export Your Logs to Snowflake and Tableau
Переглядів 374 місяці тому
While you can monitor your processes in Shipyard's holistic data operations platform, sometimes you may want to have a dashboard in your own tool to monitor your Fleets. With the Shipyard API Blueprint, you can do just that! In this video, we walk you through how you can export logs from Shipyard into Snowflake and Tableau.
Inside the Life of a Data Consultant
Переглядів 334 місяці тому
Ever wondered what it's like to be a data consultant? In this video, get an inside look at the day-to-day life of a data consultant working with Fortune 500 companies. Plus, get some interview tips for breaking into this exciting field! Timestamps: Introduction and Overview 00:00 Introduction to the Captain's Compass series and today's focus on a data consultant's life. 00:15 Reminder to check ...
Data Science Project: BMI Index Predictions Using a Python Machine Learning Model
Переглядів 2035 місяців тому
Looking for a data science project idea? Watch this video on predicting BMI index using a Python machine learning model. Get insights on healthcare data science projects and learn more about Python and machine learning. 00:00 - Introduction and Project Overview: Celebrating HIPAA Compliance 04:15 - Setting Up: Importing and Preparing the BMI Dataset from Kaggle 08:00 - Data Cleaning and Initial...
The Life of a Data Engineer: What You Didn't Know
Переглядів 1375 місяців тому
In this video, we'll take a closer look at the exciting world of data engineering. From designing and building data pipelines to working with cutting-edge AI technology, we'll give you a glimpse into the day-to-day life of a data engineer. If you're curious about the field of data engineering, or looking to pursue it as a career, this video is for you! More Captain's Compass episodes Learn Juli...
Learn Tips for a Career in Analytics Engineering
Переглядів 1976 місяців тому
Learn Tips for a Career in Analytics Engineering
Unlocking Business Success: Choosing Dashboards or Spreadsheets
Переглядів 436 місяців тому
Unlocking Business Success: Choosing Dashboards or Spreadsheets
How to Start or Advance Your Data Career
Переглядів 336 місяців тому
How to Start or Advance Your Data Career
Revealing the Insider Tips for Business Intelligence Analysts
Переглядів 1727 місяців тому
Revealing the Insider Tips for Business Intelligence Analysts
Start Using DuckDB in Python Today
Переглядів 1027 місяців тому
Start Using DuckDB in Python Today
Secrets of Shipyard Crew's New Year Goals
Переглядів 218 місяців тому
Secrets of Shipyard Crew's New Year Goals
Inside Look: Shipyard Crew's Favorite 2023 Projects
Переглядів 328 місяців тому
Inside Look: Shipyard Crew's Favorite 2023 Projects
Escaping dbt Cloud: Move to dbt Core
Переглядів 7098 місяців тому
Escaping dbt Cloud: Move to dbt Core
Data Engineering Project: Quickly Build a Complete Pipeline
Переглядів 47010 місяців тому
Data Engineering Project: Quickly Build a Complete Pipeline
Orchestrate Your Data Tools Free: Get the Details
Переглядів 9610 місяців тому
Orchestrate Your Data Tools Free: Get the Details
Data Engineering Project: Quickly Build a Complex Data Pipeline
Переглядів 10 тис.Рік тому
Data Engineering Project: Quickly Build a Complex Data Pipeline
Generative AI, BI and Analytics: Where It Is, Where It's Heading with Sonny Rivera
Переглядів 95Рік тому
Generative AI, BI and Analytics: Where It Is, Where It's Heading with Sonny Rivera
Who is Data Orchestration For?
Переглядів 174Рік тому
Who is Data Orchestration For?
Data Orchestration Demystified: A Beginner's Guide
Переглядів 472Рік тому
Data Orchestration Demystified: A Beginner's Guide
Master Two Workflows, Fast: Snowflake and External Tools
Переглядів 43Рік тому
Master Two Workflows, Fast: Snowflake and External Tools
What are Ephemeral Files, Otherwise Known as Temporary Files?
Переглядів 72Рік тому
What are Ephemeral Files, Otherwise Known as Temporary Files?
What is Data Extraction?
Переглядів 65Рік тому
What is Data Extraction?
What is Snowflake Data Unload and Why You Should Care
Переглядів 123Рік тому
What is Snowflake Data Unload and Why You Should Care
Fivetran and Jira Integration: How to Avoid Failure
Переглядів 109Рік тому
Fivetran and Jira Integration: How to Avoid Failure
What is Snowpark and Why Should I Care?
Переглядів 157Рік тому
What is Snowpark and Why Should I Care?
What Problems does Data Orchestration Solve?
Переглядів 220Рік тому
What Problems does Data Orchestration Solve?
Automate Transcribing your YouTube Videos
Переглядів 87Рік тому
Automate Transcribing your UA-cam Videos
Airflow Alternatives: Find the Right Data Orchestration Tool
Переглядів 157Рік тому
Airflow Alternatives: Find the Right Data Orchestration Tool
Improve Your Efficiency with Open-Source Low-Code Orchestration
Переглядів 86Рік тому
Improve Your Efficiency with Open-Source Low-Code Orchestration
Data Orchestration vs. Data Observability: Learn the Differences
Переглядів 69Рік тому
Data Orchestration vs. Data Observability: Learn the Differences

КОМЕНТАРІ

  • @gabrielescalante6482
    @gabrielescalante6482 4 місяці тому

    great video!

  • @diaconescutiberiu7535
    @diaconescutiberiu7535 6 місяців тому

    Such an informative video. I found it very useful. Just got a new job where i need to migrate entire Hadoop (oozie) workflows in Snowflake using dbt, with all 4 technologies new to me. This type of videos are extremely helpful. If you can make more on the modelling part with dbt, would be great

    • @shipyardapp
      @shipyardapp 5 місяців тому

      Thanks for the comment! What type of modeling content would you like to see from dbt?

  • @YEM_
    @YEM_ 7 місяців тому

    Why not create an out of the box vessel for it with parameters to fill out for GitHub details and command to run (etc)? It sounded like you were saying there's some technical challenge... But there's a ton of other tools out there integrating dbt core from a GitHub repo, with a nice wysiwyg form. I think it would go a long way for you guys... To honestly say you have a dbt core solution instead of a python tool.

    • @shipyardapp
      @shipyardapp 7 місяців тому

      Great point! We have thought about making low-code templates for these, but we decided to give the users the ability to create this for themselves. There are small changes needed for each individual database that the user needs to take care of on their end.

  • @thedatadoctor
    @thedatadoctor 7 місяців тому

    Glad to see this!

    • @shipyardapp
      @shipyardapp 6 місяців тому

      Thanks so much! Any other content you'd like to see from us? BTW, we really like your channel!

  • @HomerSimpson278953
    @HomerSimpson278953 11 місяців тому

    Dear author, thank you, very interesting! Can you tell me how to download a ready-made file with subtitles directly? Without unnecessary steps, such as uploading to the cloud, etc.

    • @shipyardapp
      @shipyardapp 11 місяців тому

      We currently do not have a way to download directly to your PC. However, you could email it to yourself and download it that way.

  • @rsedytube
    @rsedytube 11 місяців тому

    Can this flow be revered. extract data from SqlServer in CSV file to google Storage

    • @shipyardapp
      @shipyardapp 11 місяців тому

      It absolutely can. You would use the Download Query as a CSV Blueprint with SQL Server then use the Upload File Blueprint with Google Cloud Storage

  • @shipyardapp
    @shipyardapp Рік тому

    Full end-to-end project: ua-cam.com/video/IQxhutCbN7A/v-deo.html

  • @avishekchakraborty4325
    @avishekchakraborty4325 Рік тому

    How to do incremental load for a table while transferring data from snowflake to Big Query.

    • @shipyardapp
      @shipyardapp Рік тому

      Essentially you would need to setup your query to do the incremental load. If the query runs in Snowflake, it will run in Shipyard

  • @manualdevalor
    @manualdevalor Рік тому

    its given the error "pytube.exceptions.RegexMatchError: get_throttling_function_name: could not find match for multiple " when download the video

    • @shipyardapp
      @shipyardapp Рік тому

      Hello! The package that we use for downloading UA-cam videos is having issues. We are working to rectify this as we speak. Sorry for the issues!

  • @jeffmasters4879
    @jeffmasters4879 Рік тому

    Any solution to do this with twitch livestream in real-time?

    • @shipyardapp
      @shipyardapp Рік тому

      Hey Jeff! Shipyard isn't made for real-time data processing. I have built some Twitch integrations that work every 30 minutes or so to produce clips and things of that nature. You would need to use a product that can handle real-time processes to complete that unfortunately.

  • @Deepgram
    @Deepgram Рік тому

    Beautiful and intutive! This is such an incredible demo. Thanks for showcasing our transcription models. We can't wait to see what you have in store next!

  • @mrjonkane
    @mrjonkane Рік тому

    Set all this up and hope I can get it to work but getting an error from Whisper in the logs that says: openai.error.APIError: Maximum content size limit (26214400) exceeded (26416764 bytes read) To be fair the video is 1h 50m long so may simply be too big. I wonder if we could set an in/out point on videos by timecode so Whisper only transcribes shorter segments.

    • @mrjonkane
      @mrjonkane Рік тому

      I got it to work using this video (ID Yu-gWepOiS4). Looks like limit is 25MB for the audio file in Whisper. Instead of translating to another language however I'm just using a prompt to OpenAI to 'Summarize the text' (of the transcribed video). BUT this is what it provided as a summary for this video... weird 🤔 "The text talks about the benefits and drawbacks of working from home during the COVID-19 pandemic. The advantages of remote work include less commuting, increased flexibility, and improved work-life balance. However, remote work can also negatively impact mental health due to isolation and lack of social interaction. Additionally, working from home can blur the boundaries between work and personal life, leading to increased stress and burnout. Finally, the text concludes that a hybrid model of in-person and remote work may be the best solution post-pandemic."

    • @shipyardapp
      @shipyardapp Рік тому

      Super interesting. Unfortunately, we can only work in the confines of the APIs provided. You could definitely write a Python script to chop the video into pieces and roll from there.

  • @andriigorokhovskyi2709
    @andriigorokhovskyi2709 Рік тому

    Hey, I'm interested in exactly this setup! Especially with webhook and a variable; Reached out to your support

    • @shipyardapp
      @shipyardapp Рік тому

      Just got over a response! Look forward to chatting with you!

  • @Jellyg00se
    @Jellyg00se Рік тому

    Interesting - Bit of a clunky / slow method using shipyard for me though... you could create an OpenAI API token, ask ChatGPT for the Python code to write a small program that takes the file as an input to perform an API request chatGPT asking to comment the code. Then send it to postfix or whatever the Windows land equiv is and you could do this from the terminal / cmd in one line :)

    • @shipyardapp
      @shipyardapp Рік тому

      No doubt about that! We will never claim that we are the only solution to any of the use-cases we provide. We just want to make sure that people have the option to do those things in the platform if they want!

    • @Jellyg00se
      @Jellyg00se Рік тому

      @@shipyardapp It wasn't a dig, just an observation - Seems as though Shipyard might be more of a user friendly Jenkins style pipeline for your cloud needs?

    • @shipyardapp
      @shipyardapp Рік тому

      Absolutely right! We are trying to take some tasks that were seen as too technical and make them available to the masses!

  • @pierremichel-angealexandre4488

    Very important question sir , How can I translate a .txt file from a specific language to an another specific language (not English)? How do I specify the second language please ? test.txt --model medium --language Spanish --task translate Spanish --> French

    • @shipyardapp
      @shipyardapp Рік тому

      Hello Pierre! We currently do not have any translation services in Shipyard natively, however you could write a custom Python script to do that translation in Shipyard. Feel free to reach out in Intercom in the application if you need help with that script!

  • @DarienDrakee
    @DarienDrakee Рік тому

    I was having issues setting up python / whisper and everything else that goes along with it on my computer. I am not expert or even an armature when it comes to this type of stuff, but shipyard is pretty amazing. it makes things 100x less complicated. thanks

    • @shipyardapp
      @shipyardapp Рік тому

      That's great to hear DJ! I'm glad you have been able to get some processes running in Shipyard!

  • @martinltmr8590
    @martinltmr8590 Рік тому

    Can you schedule this to be sent on a monthly basis?

    • @shipyardapp
      @shipyardapp Рік тому

      Absolutely. We have scheduling and webhook triggers to start Fleet runs. You can read more about them here: www.shipyardapp.com/docs/reference/triggers/schedule-triggers/

  • @AliKhaled-kd2mb
    @AliKhaled-kd2mb Рік тому

    Can you help me to get the slack token for my channel?

    • @shipyardapp
      @shipyardapp Рік тому

      Sure! If you check out our documentation here: www.shipyardapp.com/docs/blueprint-library/slack/slack-authorization/. We walk through the steps to get the Slack token. If you have any other questions, feel free to reach out to us using intercom on our website.

  • @mostafa3282
    @mostafa3282 Рік тому

    Hi, i have an error in sending email, can you help me with this?

    • @shipyardapp
      @shipyardapp Рік тому

      Sure! Feel free to send us a message through Intercom on our website, and we would be happy to help!