The Worst Website Launch of All Time

Поділитися
Вставка
  • Опубліковано 17 тра 2024
  • With a $464,000,000 budget and a timeline of three years, CMS and company attempt to build a website. Unfortunately, it does not go as planned.
    Sources:
    oig.hhs.gov/oei/reports/oei-0...
    Chapters:
    0:00 Intro
    0:22 Part 1: Optimistic Beginnings
    1:08 Part 2: The Dumpster Fire Begins
    4:49 Part 3: Countdown to Launch
    7:48 Part 4: The Last 40 Days
    10:35 Part 5: Launch and Aftermath
    Notes:
    - at 0:54, this logo is fake, I couldn't find any logo for OCIIO (probably because it was short-lived and merged into CMS soon after)
    - at 1:22, this merge moved technical staff from OCIIO to OIS, the Office of Information Services (now OIT, the Office of Information Technology) which is a subdivision of CMS. The bulk of the staff was put into a new CMS subdivision, CCIIO (Center for Consumer Information and Insurance Oversight), which was just OCIIO renamed and operating under CMS.
    Music:
    - Tensions Run High by soundridemusic
    - Fine Dining by TrackTribe
    - Impact Prelude by Kevin MacLeod
    - Cool Vibes by Kevin MacLeod
    - Aloft by LEMMiNO • LEMMiNO - Aloft (BGM)
  • Наука та технологія

КОМЕНТАРІ • 495

  • @itsnumpty
    @itsnumpty Рік тому +3013

    The fact they brought in experts after spaghetti code was written for years and they fixed it in such a short time says a lot about their planning and competence.

    • @karmatraining
      @karmatraining Рік тому +397

      I found this quite striking too. The difference between a vast bureacracy and an actual, functioning coding shop with decent engineers and not too much red tape is night & day.

    • @andrewallbright658
      @andrewallbright658 11 місяців тому +54

      This is a plot point in Shin Godzilla (2016). I think about that movie a lot as a software engineer and as a citizen of the US.

    • @ogredev
      @ogredev 11 місяців тому +201

      I got to be part of this effort and man it was a train wreck. One of their "solutions" to fixing db connection deadlocks was to perform dirty reads with NOLOCK directives. Their upper management team was the best part. Having such little experience with dev teams and doing nothing but creating bottlenecks for the devs, they were often left out of the loop on purpose. Yeah I got years of horror stories working with these guys.

    • @BTrain-is8ch
      @BTrain-is8ch 11 місяців тому +70

      @@karmatraining Beyond a certain size most software producing shops become vast bureaucracies. Decent engineers and not too much red tape is a state that tends to vanish somewhere around the point where the engineering department's headcount goes beyond 10-20.
      The whole story has all the hallmarks of basically any software project at your run of the mill mid to large sized enterprise organization.

    • @timmy7201
      @timmy7201 11 місяців тому +73

      I've worked on two government funded projects myself, as a software engineer. Both went ... well ... terrible.
      There are just to many upper and middle-managers in our governments bureaucracy. All these people search purpose in their job, whilst there isn't one. So they fill their days with meetings and more meetings, these meetings result in new ideas, which result in project change requests.
      All these change requests result in a dev-team that goes in endless circles, without any progress at all. Due to this lack of progress, the client and/or government officials lose fate in the engineering team. This results in a massive increase of government bureaucracy, which just roadblocks the dev-team to a complete stop. Which result in less progress, which in return results in less trust, which results in more bureaucracy, etc ...
      The slower things go, the more the government officials start making their own decisions, the worse the project becomes...

  • @vittordecastro3815
    @vittordecastro3815 Рік тому +2918

    As any government project they were leaning towards the oldest platform they could find 😂😂

    • @lightning_11
      @lightning_11 Рік тому +59

      Why is this so true?

    • @swizzler
      @swizzler Рік тому

      @@lightning_11 Crotchety project managers high up that refuse to learn new things. The type of people that here in 2023 still insist on running Windows XP and ignore all the shit that breaks because they are doing that. I have dealt with people like that and it's miserable, but they're everywhere.
      There is also a type of project manager that will only allow a technology to be used if they understand it, but they are also an idiot, so you're stuck building rocket ships with Duplo Blocks.

    • @alexandruilea915
      @alexandruilea915 Рік тому +186

      @@lightning_11 Because they usually need to support some old shit that is still in use on the government computers.

    • @raptoress6131
      @raptoress6131 Рік тому +18

      Our public healthcare providers went with MUMPS...

    • @lightning_11
      @lightning_11 Рік тому +4

      @@alexandruilea915 (It was supposed to be a joke... |0| )

  • @napoleonVT
    @napoleonVT 7 місяців тому +515

    "They didn't document anything, but that's pretty standard, we can give them a pass" I've never been more insulted by something I completely agree with.

    • @peterdieleman303
      @peterdieleman303 3 місяці тому +23

      Industry standard practice.

    • @pharoah327
      @pharoah327 3 місяці тому +11

      I HATE that this is true. Ever tried to continue a project made by another team? When you have no contact with that team and no code is commented plus no diagrams or any documentation is given, good freaking luck! I just got finished with a project like that. It was hell.

    • @fredjones5698
      @fredjones5698 3 місяці тому

      @@peterdieleman303the great thing about this industry standard practice is that it actually applies to all industries ever. No one ever documents shit. Humans are lazy

    • @Powderlover1
      @Powderlover1 3 місяці тому

      That’s a lot of extra work

  • @timmy7201
    @timmy7201 11 місяців тому +848

    As a software engineer, I've worked on a bunch of government funded projects.
    The last project was rather small, so I estimated that it would take 2 to 3 months to deliver. The project required a specific type of IOT wireless access point, which wasn't yet installed. So we scheduled a meeting with government officials, in order to discuss an optimal installation point for this new antenna. They agreed to install the antenna themselves, so we continued developing the backend of the project.
    We completed the dev-work in less than two months, ahead of time. So we contacted the city officials, to share pairing codes for the antenna. They informed us that the antenna hadn't been deployed yet, so we had to wait.
    We contacted them 1 month later, same story.
    Again 2 months later, still no antenna.
    Another three months passed, still no antenna.
    We called them back after 6 months, begging to deploy those IOT devices to free up some office space. Explaining that we could active them remotely, once they installed the antenna. They agreed, we deployed the IOT devices, then everyone forgot about the project.
    It took the government 2.5 years in total, to install the antenna. Which is about an equivalent amount of work, as installing an wifi AP. They then asked for a financial compensation, claiming that we delivered the project two years behind schedule.
    A government funded project, in a nutshell... KEEP AWAY!!!

    • @chochoize
      @chochoize 8 місяців тому +68

      Yup sounds WAY too familiar, never again, thats for sure.

    • @meghanachauhan9380
      @meghanachauhan9380 8 місяців тому

      arre you kidding? Almost every single government contractor I've seen over quotes the living hell out of everything. A 600 worth equipment costs 6000, 6000 costs 60000. They just pay the officials in charge and the tax agencies and the officials don't care either. I mean it's not their money, it's public money and how many people even bothered to read the constitution? People kill each other for government contracts because they're legal licenses to steal. After all if everything's public money and the public doesn't care.....

    • @cones914
      @cones914 7 місяців тому +10

      did the company pay compensation?

    • @timmy7201
      @timmy7201 7 місяців тому +143

      @@cones914 No! The company I worked for had a long enough paper trail, to proof they where not in fault.

    • @leiregyp5814
      @leiregyp5814 6 місяців тому +56

      @@timmy7201 good, its absolutely sad to see how a government is so good at wasting money then blames it on others

  • @evanbelcher
    @evanbelcher 8 місяців тому +262

    This project took 3 years. I genuinely feel that if they hired a single skilled full-stack engineer, a graphic designer, and a ux designer, paid them good money for those 3 years, and gave them access to domain experts to set the requirements, they could have easily cruised to the finish line on this.

    • @NodokaHanamura
      @NodokaHanamura 6 місяців тому

      But that would require Government contracts to not, y'know, be bloated clusterfucks that only upper management would love.

    • @randyekrer431
      @randyekrer431 5 місяців тому +25

      i mean, functionality wise, yes, it would work better. but the tech requirement sheet they provide would be prob 2000 pages, given the bureaucratic crap, so f that shit.

    • @syte_y
      @syte_y 4 місяці тому

      A friend of a friend worked on healthcare.gov. I think the level of effort is being severely underestimated. The requirements kept changing because the people having the project built didn’t even know what they were building.

    • @alexwakeman8321
      @alexwakeman8321 3 місяці тому +10

      You expect the government to do something correctly? That's a high bar

    • @tahaak
      @tahaak 3 місяці тому +14

      Every time I hear about such projects, how long they took and how expensive they were I‘m thinking to myself that I could have done it in a third of the time for 100 times cheaper and it would work better. Sometimes I feel that the government deliberately does everything wrong in software projects.

  • @lucaspepe7294
    @lucaspepe7294 Рік тому +768

    "The production launch is the end-to-end testing". It gives me the same vibe as the meme of the dog that says everything is fine while sitting in a house on fire. 🤣

    • @narnigrin
      @narnigrin 11 місяців тому +40

      That line gave me a literal cold sweat for a second. When I hear that from a superior in my line of work it's my signal that it's time to polish my LinkedIn profile

    • @akshaypendyala
      @akshaypendyala 11 місяців тому +1

      😂😂😂

    • @Mavendow
      @Mavendow 9 місяців тому +3

      They thought they were a gaming startup. Unfortunately, EA wasn't looking to buy and destroy a health insurance website.

    • @pyromancy8439
      @pyromancy8439 8 місяців тому +5

      Once in a while I entertain myself by pushing untested code straight into production. By "untested" I don't mean not tested for security, bugs or load capacity, I mean I'm not even sure it compiles.

  • @TankorSmash
    @TankorSmash Рік тому +118

    "Only down 10% of the time"

    • @everyhandletaken
      @everyhandletaken Рік тому +8

      Underdelivering is important, as they then justify their existence by wasting even more time & money, resulting in management receiving salary bonuses.

  • @nielsvanderveer
    @nielsvanderveer Рік тому +994

    This is basically every pitfall that a project may encounter in 13 minutes. Seriously impressive to encounter all of this in one project 😂 Especially the idea that more engineers will speed up the delivery makes clear the managers had no clue about software engineering.

    • @DamianTheFirst
      @DamianTheFirst Рік тому +58

      exactly. That's the thing that bothers me the most - how managers could even try to manage dev team if they have no idea about how do developers work? And it's the case in most of companies I've heard of.

    • @jkf16m96
      @jkf16m96 Рік тому +36

      Yeah, usually a good project just need a bunch of seniors, one for each area.
      One senior in frontend,one in backend with another one that knows about databases, the DBA, if needed.
      One for security practices, just to make sure each one knows which data should leave their layer.
      The leader who knows exactly what is going to happen in the project, how it should be built, assisted with a software architect.
      This would be the ideal for a medium sized project, or even big sized project if every each one of them is truly a senior.

    • @BrankoDimitrijevic021
      @BrankoDimitrijevic021 Рік тому +19

      As Fred Brooks famously said: adding more engineers on a project that is late will make it later.

    • @karmatraining
      @karmatraining Рік тому +38

      It's like thinking that adding more women can reduce a pregnancy from 9 months to 1 month.

    • @peteheatb3
      @peteheatb3 11 місяців тому +8

      Its a situation thats way too common - ive seen this in utility company projects, one of which should have utilized two developers for 3 months time. When we arrived they had spent 6 months with 28 developers across three timezones, complete shitshow to approach projects that way.

  • @dalar2
    @dalar2 Рік тому +616

    Seriously you make some of the best software documentaries, I love it!

    • @robschn
      @robschn Рік тому +9

      For real! They're my fave

    • @gblargg
      @gblargg Рік тому +25

      They're goofy funny even the silly animations are full of technical details. And plenty of explosions.

  • @jinyuliu2871
    @jinyuliu2871 11 місяців тому +249

    The US visa application website, also crated by CGI, is probally the worst website I ever had to deal with. It has a number of pages which you would need to complete for the applicaion, however they have an extreamly short time out of maybe 5 minutes or so before they would boot you out of the system. Although you can save your progress, saving is only possible once an entire page is completed. It was extreamly furstrating when you go dig though the draws for a document needed to complete a question near the end of a page, only to return to see that the system has timed out and anything filled in on that page is lost. It has gotten to the point that I re-entered questions for a page so-many times due to the time out that I memorized the entire page.

    • @milesmartig5603
      @milesmartig5603 9 місяців тому +20

      @@Rubicola174 or make it open source so that others don't have to choose between endless suffering or paying money for a 3rd party app just to apply for a visa. Or the government could make the improvement, but whatever.

    • @sebastiancarreira5832
      @sebastiancarreira5832 9 місяців тому +24

      That's very possibly by design thought. Make it as hard as possible to apply for a visa, you will have to give less visas.

    • @meghanachauhan9380
      @meghanachauhan9380 8 місяців тому

      @@milesmartig5603 hey it's not my fault your elected government is dumber than a 12 year old. Now pay up

    • @baumdf9134
      @baumdf9134 8 місяців тому +8

      Are you sure you were applying for a visa and hadn't stumbled upon the time trial mode of visa simulator 9000

    • @ogonbio8145
      @ogonbio8145 8 місяців тому

      i woulda made an ahk script to jiggle my cursor or something

  • @andrewchang7194
    @andrewchang7194 Рік тому +92

    It’s funny how it took hundreds of people to build this and still fuck it up, but nowadays, this is probably a system design question for an entry level position at a tech company lol

  • @Nadia1989
    @Nadia1989 Рік тому +78

    Shoutout to the government project I worked in 2017, I've heard they're still using an IIS with php 5.6 with the vendor directory commited to the repo.

    • @bryanhoffman4331
      @bryanhoffman4331 Рік тому +17

      At least they have a vendor directory and didn't just paste files from other libraries in the src directory.

    • @narnigrin
      @narnigrin 11 місяців тому +5

      How to make a developer cry in less than thirty words

    • @zombie_pigdragon
      @zombie_pigdragon 11 місяців тому +3

      It's not ideal, but committing vendor is "fine" in practice. It's similar to a monorepo setup, which is acceptable, and it does come with a benefit in preventing some possible supply chain issues.

  • @sortsvane
    @sortsvane Рік тому +273

    You should consider launching a patreon or memberships... The content quality is top notch.

    • @Aunarky
      @Aunarky Рік тому

      ^^^

    • @manojramesh4598
      @manojramesh4598 9 місяців тому +1

      Don't need let knowledge be free

    • @noobiii
      @noobiii 8 місяців тому

      @@manojramesh4598 patreon/memberships are optional

    • @NoxDolore
      @NoxDolore 8 місяців тому

      ​@@manojramesh4598the knowledge is free here but the creators should still be able to have support 😊

    • @jimiyu.
      @jimiyu. 3 місяці тому +1

      @@manojramesh4598well the video creator is going to need to make a living somehow

  • @leodler
    @leodler Рік тому +232

    That Office of Consumer Information and Insurance Oversight logo just doesn't sit right with me lol

    • @kevinfaang
      @kevinfaang  Рік тому +130

      its fake (I couldn't find a real logo) 😔

    • @radiosification
      @radiosification 11 місяців тому +1

      What time in the video is that at? I couldn't spot it

    • @davidddisjesus
      @davidddisjesus 11 місяців тому +8

      @@radiosification I managed to find it at 0:53, it's the logo on the right

    • @radiosification
      @radiosification 11 місяців тому +1

      @@davidddisjesus Ahh I see it, thank you

  • @AndreiTache
    @AndreiTache Рік тому +266

    As someone who doesn't know anything about web dev, I find it so ridiculous how over complicated everything seems to be. There is no way a basic webpage and form colector should require 50 services built by 10 different teams

    • @LKRaider
      @LKRaider Рік тому +34

      They obviously needed wEbScALe !

    • @robinspanier7017
      @robinspanier7017 Рік тому +145

      as a webdev i can say: no, it realy does not. projects like this would be solvable by a single team.
      the decisions made along the way were the problem.
      i have seen it myself. a little change from 50 shown characters to 500 can easily cost 100k when you have badly written software, no way of testing, different contractors that are not willing to help you and multiple instances of ppl having to accept the execution of the change.
      whats even worse is that the good players quit at this point and all left is a bunch of losers operating.

    • @parabolicpanorama
      @parabolicpanorama Рік тому +26

      most of any modern website you use is built with many different libraries written by developers all over the world. no one writes the whole thing themselves because someone else has written some piece of code and it's much better than you could hope to. it's easier and smarter to just integrate that into your workflow. a "basic" modern looking web page would be a pain to write all from scratch, especially if you want to serve many different people.

    • @mipmipmipmipmip
      @mipmipmipmipmip Рік тому +41

      A issue is it has to connect to all insurance providers. It's not a scaling startup, the audience is millions of people from day one. Plus, scope needs to be determined from scratch.

    • @MrSquishles
      @MrSquishles Рік тому +27

      worked on it a few years after the time period in this video, long story short it's not that simple, hundreds of thousands of lines of backend code levels of not that simple.

  • @xplinux22
    @xplinux22 Рік тому +148

    That was an astounding trainwreck from beginning to end! I lived in Singapore for a few years prior 2020, and I was spoiled by the quality and slick interfaces of all the government technology over there, created by their in-house SWE agency GovTech. Everything from the SingPass app (for national ID and SSO) to their tax portal was extremely solid and modern looking, rivaling most commercial apps. It's so fucking sad how far behind the US government is, when it comes to software engineering.

    • @johnpaulgeorgeringo2329
      @johnpaulgeorgeringo2329 11 місяців тому +16

      Yeah, I don't know why most of their resources goes to weapons and wars

    • @REAL-UNKNOWN-SHINOBI
      @REAL-UNKNOWN-SHINOBI 11 місяців тому

      ​@@johnpaulgeorgeringo2329It's so that the United States government can bully other governments into loving them.
      Even though it just shows that you are corrupt a****** who will use violence and force to make friends.

    • @holy3979
      @holy3979 11 місяців тому +20

      It's mostly down to how our government is structured here in the states. It's structured in such a way that making changes takes a very long time and a lot of political will, for both good or bad. This prevents a single administration from abusing power, however at the same time it means that the government lags far behind when it comes to modern rapidly developing technologies.

    • @xplinux22
      @xplinux22 11 місяців тому +18

      @@holy3979 I'm not so sure about that. Both NASA and NOAA do some incredible engineering and scientific work today, and so does the DoD when it comes to aerospace and IT (just look at the invention of GPS and the early Internet for past examples), but somehow, the US just sucks hard at software.
      *EDIT:* Forgot to mention NIST, the NSA, and the U.S. Naval Research Laboratory as prime examples of the US government successfully developing and standardizing some incredibly advanced computer tech, e.g. Tor, Ghidra, and several key cryptography standards.

    • @Lightningflamingice
      @Lightningflamingice 11 місяців тому +7

      @@xplinux22 This is probably because of how good the software industry is in the US. It's not the US public that sucks at SWE, it's the US gov't, and the reason for that is that the gov't can't afford to match the good compensation a tech company will provide, which leads to all the talented people (both in technical and non-technical skills like communication and management) being drained towards private industry.
      Like you said, NASA and NOAA are top-notch, and that's because for the top-notch hardware engineers, these are some of the best places you can work at because only government can afford the high upfront costs to doing things like maintaining a successful aerospace program.

  • @MCasterAnd
    @MCasterAnd Рік тому +94

    You should really do a video on when Helse Midt (Norway's healthcare organization that covers the middle area of Norway) chose to change their journal system.
    They started with a bidding process. All vendors welcome. Pretty early in the bidding process however they decided to disqualify DIPS, who was delivering the same journal system to Helse Sør-Øst, Helse Nord and Helse Vest - the three other govt. healthcare organizations in Norway. Yes, they really disqualified the one piece of software that was used literally everywhere else. Who did they go with? Well, good'old american EPIC - and they chose to call the new system "Helseplatformen". Did it go well? NOPE.
    It has been a shitshow from day one. Most notably, 16 000 critical letters to patients were discovered to not have been delivered, which delayed a whole boatload of appointments and also put some patients lives at risk (Helseplatformen blamed this on user error - yes, really, 16 000 missing letters was the result of a user error made by several hundred users was to blame on the users, not the system). After this, the director of Helseplatformen stepped down due to the huge wave of criticism following the reveal of these issues.
    One doctor actually saved a patient life after being unsure about wether or not a letter was sent. He asked a colleague to directly contact the patient to ensure he got the information he needed. The patient had a blood clot. Had the hospital not reached out to him directly, he could have died.
    Currently, 44 individual cases have been confirmed where patients have not received the help they needed in time, directly as a result of Helseplatformen.
    In a survey conducted among the staff of one of Helse Midt's largest hospitals, 27% of the nurses were considering quitting due to the issues.
    The feature for referring patients to another hospital, which is a critical feature of any hospital journal system, was non-existent when the system was first implemented. After several months this feature was finally implemented, but it caused a massive amount of manual labour for the hospital staff - who are already understaffed.
    Some hospital directors are quoted in meetings discussing the backlash from doctors and healthcare workers as "echo chambers", "hate groups" and "whining from angry doctors" - which isn't really helping the issue. The healthcare on their side are claiming that they are providing the directors with "constructive criticism" about legitimate issues with the system.
    In addition, the price for the new system has increased by 35%, from 3.7 billion kroner to 5 billion kroner - and it has been implemented in less than half of the healthcare institutions in the region.
    UX/UI wise it's also a complete mess. It's one of the worst pieces of software I've ever seen in modern times. If you search for Helseplatformen on google you can see their take on inputting a date/time in the system. It's a complete scandal.
    Now, a ton of doctors and healthcare workers have protested against Helseplatformen - with some healthcare institutions outright refusing to change to the new system, instead opting to use the old one. In the meantime Helse Sør-Øst, Nord and Vest are looking at this and wondering why the hell they didn't go with DIPS, so they could all use the same system and be able to share journals across all hospitals...

    • @tangiblewaves3581
      @tangiblewaves3581 11 місяців тому +3

      Really awful story 😢 but quite frequent the way how federal software projects work. It's a real mess, and I wonder why nothing is learned from this. There are brilliant tech companies out there making immensely powerful systems; why can't this knowledge not be brought into federal software??? 🤷🏼‍♂️🤷🏼‍♂️

    • @Sammysapphira
      @Sammysapphira 10 місяців тому +1

      Wow I don't think I've ever seen a worse ui. The date input looks like what a day 3 high-school student would make in a programming class.

    • @macmarc6661
      @macmarc6661 10 місяців тому +4

      Can anyone share a link to the date picker lmao

  • @ttuurrttlle
    @ttuurrttlle Рік тому +164

    This is very interesting, but I feel like I still have no clue how this actually could have happened on the ground. Like yeah, there was no decisive leadership, stupid managerial practices, old technology, changing requirements... it does sound like a perfect storm for things to go badly, except that's kinda par for the course in professional software development.
    It does sound like everyone managing it was at fault from contractors to government officials because of various bad decisions and the project was a nightmare to work on. It sounds like everything boiled down to terrible management, but I'm still sorta surprised about how that happened to such an important project. It's not like the federal government hasn't made working websites before...
    I would have liked to see more specifically how that GUI-generated code played a part, cause I can see that being a problem. I've definitely felt restrained and exasperated by bad code I could not change.

    • @asii_k
      @asii_k Рік тому +21

      I'm curious about the out of control db queries too, think I'll check the report and see if it has anymore details on that

    • @shimadabr
      @shimadabr Рік тому +12

      Incompetence compounds over time.

    • @jkf16m96
      @jkf16m96 Рік тому +30

      The main problem is almost always the management.
      If they tell you "this project is going to use X" you tell them "X is truly outdated and almost no one uses it" and they just shush you.
      Well, no one knows about X and now the whole team has to learn X with outdated documentation or lost documentation.
      This has happened to me a few times, when a boss asked me to use X or Y, I told him how outdated it was, then everything just slow downs a lot because there is hardly documentation and just some online posts from 2009 or 2010

    • @thewhitefalcon8539
      @thewhitefalcon8539 Рік тому +12

      This is basically just what happens when there isn't leadership. The point of a leader is to make sure everyone is working together to achieve the end goal, not doing their own thing based on their own version of the end goal that doesn't fit together at the end.
      (Continuous integration and testing are just two specific technical mechanisms that ensure everyone's pieces fit together)

    • @thewhitefalcon8539
      @thewhitefalcon8539 Рік тому +10

      @@asii_k I assume it's just the N+1 queries problem: because you put more effort in making the code "clean" than fast, you have a "get whatever from database" function and the only way to get 100 whatevers is to call it 100 times instead of asking the database for all 100 whatevers at the same time.

  • @bvd0
    @bvd0 Рік тому +25

    5:42 This notepad scene made my day.

  • @rebelcat_1261
    @rebelcat_1261 11 місяців тому +18

    "The production launch is the end-to-end testing." I will now be incorporating this philosophy into all of my future endeavors.

  • @gmtflex
    @gmtflex 3 місяці тому +3

    You know, this video showed me that even though I only have one more day to complete my project in python, and even though I have only like 30% of it complete I can rest assured that what I am doing is standard practise in the field. God how I love being a programmer.

  • @MHX11
    @MHX11 Рік тому +16

    Your humor with the editing is amazing

  • @TheRossMadness
    @TheRossMadness 11 місяців тому +11

    Everything that happened in this scenario is covered in "The Phoenix Project". That book still holds up and I wish more people in our government would read it.

  • @paulosullivan3472
    @paulosullivan3472 Рік тому +53

    As someone who works in an agile change environment listening to you describe it as what it theoretically is supposed to be was kind of hilarious. In practice the daily stand ups are a painful mix of the product owners chastising people for not working faster, people saying "yup still working on that thing which I already said in the last three meetings" and the rest just trying to avoid saying anything which might put them in the product owners firing line. The method of controlling work on the kanban board is disorganised and communication with the business is essentially cut off by the whole agile process. The only time it works well is when you have someone with enough clout and a modicum of common sense to organise the work on spreadsheets outside of the whole process and actually get some overarching control.

    • @P4INKillers
      @P4INKillers Рік тому +19

      If your dailies consist of product owners chastising you or others over time spent, you quit.
      Simple as that.

    • @AppleGameification
      @AppleGameification Рік тому +8

      Why are product owners in scrum?

    • @narnigrin
      @narnigrin 11 місяців тому +1

      @@P4INKillers Or fire the product owner, but since that's not something an individual dev has any power over yeah I agree at that point you start sending out CVs

  • @probag8414
    @probag8414 Рік тому +16

    Do the Phoenix Pay System next! You want horror story nightmare, that's the place to find one.

    • @MrSquishles
      @MrSquishles Рік тому

      happen to also have en.wikipedia.org/wiki/CGI_Inc. working on it? i went through a few years getting on gov contracts after them, lot of clown code.

  • @Stealth86651
    @Stealth86651 Рік тому +10

    These videos are awesome, thank you so much for making these. Was also thinking, I'd be totally down to listen to this via podcast as well, but that's a lot of extra work on what I imagine is already a ton of work. Thanks again, the effort/content is really appreciated.

  • @TheShnitzel
    @TheShnitzel Рік тому +15

    Man, you're a really great storyteller and your videos are truly awesome.
    I'm glad I randomly stumbled into this channel. You deserve way more recognition!
    As an engineer myself I find these videos very interesting and valuable. Keep 'em coming!

  • @debodays
    @debodays 10 місяців тому +18

    "first they didnt document anything but that was a standard, so we can give them a pass here", this line just cracks me up😂😂😂. more or less standard across most of the companies😂😂

    • @WaylandYT
      @WaylandYT 9 місяців тому +2

      Painfully true.

  • @Chromana
    @Chromana Рік тому +7

    I discovered your channel today on the way home from work and I've already watched all your videos. Please keep it up. I hope your ad revenue allows you to make them more often.

  • @crue5191
    @crue5191 Рік тому +3

    it's amazing that such a fresh video has CC already. thank you!

  • @shimadabr
    @shimadabr Рік тому +15

    Wow, doubling the headcount with 3 months to deliver the project. Understandable, it's not like that's a well documented mistake since the 70's or something... haha

  • @douglasmasho2324
    @douglasmasho2324 3 місяці тому +5

    Meanwhile an Indian UA-camr can casually build that in a week

  • @XxHomerSimpson91xX
    @XxHomerSimpson91xX Рік тому +6

    One of the most underrated UA-cam content creator. Hilarious and really informative!

  • @user-ox8lu8cw2l
    @user-ox8lu8cw2l Рік тому +3

    Love these videos. Been waiting for you to upload again.

  • @jhonyortiz5
    @jhonyortiz5 Рік тому +17

    This video is great. Love the style. I really appreciate that it's still going into depth to an extent but trying to keep it accessible.

  • @syte_y
    @syte_y 9 місяців тому +4

    I feel like the all star team had an advantage where the business rules were more well understood. If you talk to a vendor who doesn’t even understand the project how could they possibly convey it to you to build. Also involving so many teams is just awful. Makes comms so much more difficult. The government likes over complicating anything.

  • @Salted_Potato
    @Salted_Potato 11 місяців тому +1

    The amount of times I have to pause in your videos are insane, but in a good way. I love the memes / articles curated into the video.

  • @eddydude100
    @eddydude100 11 місяців тому +3

    I just wanted to leave a comment to say that your videos about these tech disasters are brilliant. Your six most recent videos have all been so interesting and very well produced. I'm sure if you keep putting out grerat content like this then the algorithm will reward you handsomely and your channel will grow very quickly. Keep up the great work!

  • @nwrocketman6438
    @nwrocketman6438 Рік тому +17

    This is an excellent video. Keep up the good work!

  • @GbpsGbps-vn3jy
    @GbpsGbps-vn3jy 11 місяців тому +5

    In our country there was the same hi-tech attempt to enable parents to enroll their children in kindergartens. It was fiasco, system was down on day one, and every second kid was not in the lists. At it repeats each year :D

  • @TheEggman888
    @TheEggman888 Рік тому +62

    Your videos keeps reinforcing the idea that you don't need hundreds of people to develop software even for national use, if anything adding more people create more communication problem which lead to wasting even more time and money. If the people at the government had chosen instead to go with a smaller team(20 people at most), they wouldn't have encountered all these issues. It would have cost significantly less money and it would probably have been delivered on time. They could even have hired multiple teams to develop multiple concurrent version of the website and chosen the best out of them.

    • @crissd8283
      @crissd8283 Рік тому +23

      That isn't the government way. Throwing more money and resources at something that is failing is always the governments course of action. The contracts make more money, the more incompetent they are.
      I agree, hiring 20 really good developers is much better than hiring 100 bad developers. But that isn't the way the government sees it.

    • @mr_confuse
      @mr_confuse Рік тому +24

      @@crissd8283 Honestly, even 20 bad developers should mostly be better than 100 bad developers lol

    • @oliverford5367
      @oliverford5367 11 місяців тому +4

      This is known as the Mythical Man Month. Adding more people creates complexity, which can slow things down.

    • @rhas356
      @rhas356 11 місяців тому +4

      Particularly since ultimately it was "just" a large-scale insurance broking website. There are dozens of pre-existing ones - it wasn't needing to do what certain government projects must: something new that also reaches everyone,.

    • @opfipip3711
      @opfipip3711 2 місяці тому

      jeah, some of the most amazing and solid software used by millions of people is written by tiny teams or even solo devs. The entire developer team at f**king Adobe is only around 30 people. An entire PC fleet management system was written by a few German teachers over the curse of a few years and is now used in **many** schools across Germany.
      DXVK, the library translating directX to vulkan and embarrassing both AMD and Intel, by, in many cases, substantially outperforming their native drivers, was originally written by a single dev, and is now maintained by only 2 devs. It has since been included in Intel's arc drivers to fix the severe performance problems they had at launch.
      Quality software IMO requires only 4 things (for the coding part, i'm can't speak about the other things):
      A clear *vision* and focus.
      At least *one* *good* and, perhaps even more important, *motivated* *programmer*.
      And *time* to improve after an preferably soon usage start.
      Communication between everyone involved (this point is easier the fewer people work on the project)
      if the vision is missing, you will produce something, but end up with something unusable, and possibly useless.
      If you don't have a single good and motivated programmer, it will end up a buggy and mostly unfixable product, no matter how great the vision.
      If you don't give it time (in use) the program will either be unfinished or inadequate.
      If the Communication is failing, all three of them can happen. (bad communication costs time, robs motivation, and prevents a vision from driving the team forward)
      I think most software fails on the Vision or Communication front. Especially government or large company software.

  • @nessitro
    @nessitro Рік тому +7

    These documentaries of yours are exactly what I was looking for in terms of entertainment; hilarious and informative

  • @thewhitefalcon8539
    @thewhitefalcon8539 Рік тому +4

    Frequent changes is basically the point of processes like Scrum - you still gotta pay for your changes, but at least the process can accommodate them. And you still have to eventually figure out what you want if you want it to ever get done. And you can't set a deadline if you don't know what you want (but you can still try to build it).

  • @atirutwattanamongkol8806
    @atirutwattanamongkol8806 7 місяців тому +1

    The amount of miscommunications here is even more amazing than Operation Viking

  • @mr.familiar1136
    @mr.familiar1136 11 місяців тому +24

    Wow, Terremark was the most efficient one in this whole story and all they did was double it and give it to the next person.

  • @karmatraining
    @karmatraining Рік тому +8

    You should do one of these on the Queensland Healthcare Payroll systems debable. Similar amount of money was wasted, maybe more, and it took down a whole State government here in Australia. Absolutely great business case study on how NOT TO run an IT project.

  • @skyhappy
    @skyhappy Рік тому +6

    Your meme game is god tier...I kneel

  • @mikadopen4809
    @mikadopen4809 11 місяців тому +2

    Software Design teacher showed this too our class as a case study and bad example, keep up the good work!

  • @alexisdamnit9012
    @alexisdamnit9012 9 місяців тому +1

    I work at a big government contractor (formerly a data scientist at a tech company). I can confirm that government contractors are as ineffective and inefficient and bureaucratic as government agencies. It’s really bad. People here barely know how to write code.

  • @blaketomlinson3915
    @blaketomlinson3915 9 місяців тому

    your visuals during any of your videos are amazing and hilarious!😂

  • @Siyual
    @Siyual Рік тому +6

    As a career DBA, I love the database-focused nature of these documentaries.

  • @Spokeek
    @Spokeek 11 місяців тому +2

    I love the way you tell those tech stories. Really cool to follow

  • @mawcus7132
    @mawcus7132 8 місяців тому +1

    Your videos are dank AF. Keep up the great work🤙

  • @Hazanko83
    @Hazanko83 Рік тому +5

    An entire 6 people were able to successfully sign-up on day 1? 100% guaranteed at least SOMEONE got fired for making the system too easy to use.

    • @abhaynath5833
      @abhaynath5833 10 місяців тому

      It was only 1 person who made six different accounts.
      He was Chuck Norris 😂😂

  • @karstenkunneman5219
    @karstenkunneman5219 Рік тому +24

    I'm not a web developer, so maybe this is a stupid question, but how could a website whose only purpose is to allow people to buy insurance cost nearly a billion dollars, let alone the nearly half a billion dollar original budget?

    • @tylerpeterson4726
      @tylerpeterson4726 Рік тому +19

      I think it was all the change requests. If leadership says to do A, then later says A is wrong, do B, then all the money spent making A was wasted.

    • @DamianTheFirst
      @DamianTheFirst Рік тому +21

      I'm not a dev (yet) but I guess it's not only a website. The website is what user sees but devs also need to create an entire backend. It's similar to frontoffice and backoffice - as a customer you'll never see backoffice but it is needed to provide smooth operation.
      Such service relies on authentication based on some gov't services and needs to register your actions in another gov't service. Yet (as a noob) I still believe that the budget for this could be at least 5x lower and it would be sufficient for creating a decent system. It's just too many managers taking too high wages for what they are doing (i.e. creating chaos and disruption)

    • @alemswazzu
      @alemswazzu Рік тому

      It's the Government. "Other people, spending other people's money on other people".
      Plus I believe the contract was given to a friend of Michelle Obama's, not a well known tech company.
      No website, should ever even cost close to that much money.

    • @AmrXcellent
      @AmrXcellent 11 місяців тому +4

      welcome to gov inefficiencies... think how many gov employees were hired for how much time and how many sub contractors were involved and how many times these requirements were changed. Also bringing in top tier talent, doubling resources, ... all these things last min cost more.
      Not to mention that the amount mentioned doesn't mentioned what is really covered. it could be the cost for the website for 5-8yrs with support and staff. Again, not justifying the ridiculous amount of $ for what essentially is a website (and not a very complicated one at that) - it is not like a brokerage that needs to do transactions in real time or an airline booking website that needs to search from airlines in realtime from all over the world and be responsive.

    • @REAL-UNKNOWN-SHINOBI
      @REAL-UNKNOWN-SHINOBI 11 місяців тому

      Why I don't get it why did they have to rebuild the damn website from the first one.
      The first one they had worked.
      Just because it didn't let you put in payment information or sign up doesn't really matter, you're going to have to call the insurance company anyways when something doesn't work correctly.
      For a website that was supposed to make getting health insurance easy was an extremely difficult task and a big waste of time, and just convinced people just to shop around by calling everyone that they knew and to use Google and reddit.

  • @dany_fg
    @dany_fg 6 місяців тому +2

    Me after a good night sleep: "Productivity increased by 9000%"

  • @mc.ivanov
    @mc.ivanov Рік тому +2

    As an software engineer specialised in putting down fires, your channel is just the best. Thank you.

  • @AccurateBurn
    @AccurateBurn Рік тому

    Dude just watched all five of your technical documentaries, they are so entertaining. This one is especially triggering though haha

  • @vincentvanhoven3486
    @vincentvanhoven3486 7 місяців тому +2

    Delaying security testing to 6 months after launch? So, 6 months for malicious parties to exploit any possible vulnerabilities. That seems pretty bad.

  • @Michael-ri8sg
    @Michael-ri8sg 11 місяців тому +3

    "They slept in nearby hotels, while working 24 hour shifts" 🤣

  • @sill
    @sill Рік тому +2

    you're my favorite new channel.

  • @VincentSaelzler
    @VincentSaelzler Рік тому +3

    Love that the investigation from the government actually got put to use as a reference for this video!

  • @BenMclean007
    @BenMclean007 9 місяців тому +3

    This sounds exactly like my experience with government projects in Australia. Not all of them are like this, but I was in the shit house dept where everyone was trying to leave.

  • @xorxpert
    @xorxpert 3 місяці тому +1

    as an independent fullstack developer, this was absolutely depressing to watch

  • @SaudBako
    @SaudBako 9 місяців тому

    This video is scarier than most horror movies lately.

  • @RSZA011
    @RSZA011 Рік тому

    I love your channel !! such incredible content . MOAR!!!!

  • @dybdab
    @dybdab Рік тому +6

    Please make more of these type of videos.

  • @samsonsoturian6013
    @samsonsoturian6013 8 місяців тому +1

    It's actually a recurring theme for government programs to be effectively leaderless. Often the guy in charge is a politician who is preoccupied with politics and he just rubber stamps proposals since he isn't really involved in operations. Likewise, sometimes certain tasks fall under several agencies at once so progress moves at a snail's pace as any given file must go on a trek between as much as dozen agencies before being finished

  • @ravenbarsrepairs5594
    @ravenbarsrepairs5594 8 місяців тому +1

    Why would anyone expect any gov't to actually be capable of doing what it's designed to do?

  • @ohioplayer-bl9em
    @ohioplayer-bl9em 9 місяців тому +1

    The launch of the website was exactly how the entire ACA went and is still going. It’s a pos law

  • @SilverDollarSamuel
    @SilverDollarSamuel Рік тому +2

    Literally art Kevin. Keep it up!

  • @aaaaanh
    @aaaaanh 7 місяців тому +1

    imagine being the country that has the (in)famous silicon valley but fails to build a website

  • @FrozenMilkOnACloudyDay
    @FrozenMilkOnACloudyDay 9 місяців тому +1

    Your videos are fantastic, cant wait for more

  • @OldestHouse
    @OldestHouse 9 місяців тому

    the CS channel i always wanted

  • @hadipawar2539
    @hadipawar2539 8 місяців тому +1

    2 years for a an account creation popup that doesn't even work is exactly how i imagine govt projects would work.

  • @3rdalbum
    @3rdalbum Рік тому +2

    Directors have tenures of less than a year? That's not far off normal for my government directorate.

  • @fazzitron
    @fazzitron 9 місяців тому +1

    That joke about to documentation got me 😂

  • @narnigrin
    @narnigrin 11 місяців тому +4

    I write code professionally.
    This video made me want to cry.
    How could they possibly do *everything* wrong? It's like a fucking bingo card of how never to manage a software project. HOW!?

  • @Fredrovicius
    @Fredrovicius Рік тому +3

    I signed up back in 2013 and didn't get coverage, 9 years later I tried to login using the saved information and they had removed the security question I used so I was no longer able to recover my account. I called support and tried for two hours to recover my account. After 10 years - still no healthcare, they told me I have to have my wife call in as I'm now under her account but we do not and have never had coverage. It's great to know they were poorly mismanaged - when I signed up in 2013 the forms I was supposed to fill out were not available yet so I obtained an account but could not complete the setup. I was told to keep checking back.

    • @eleventy-seven
      @eleventy-seven 11 місяців тому

      California got fed up and did their own Covered California. Surprisingly, it works. Ive been signed up since the second year and saved a fortune.

  • @ryanheal89
    @ryanheal89 6 місяців тому +2

    This sounds pretty much like the project at my work.

  • @Chichi1612_
    @Chichi1612_ 9 місяців тому +1

    I love the car helicopter anology, "I though we were making helicopers" had me laughing, aswell as that car and helicoper almagation

  • @nickcocks
    @nickcocks 11 місяців тому +1

    Please keep these videos coming ❤❤

  • @guerra_dos_bichos
    @guerra_dos_bichos 7 місяців тому +2

    the good part is that this being a government project, it's all documented, but this is all too common in the private sector too

  • @vvidalftw
    @vvidalftw 11 місяців тому +1

    I had to subscribe to your channel. Definitely top notch IT content HAHAHA

  • @Kyun9432
    @Kyun9432 Місяць тому

    The delivery on this line was amazing 12:18 "EIDM was replaced by the Scalable Log-In System which was more scalable at logging in..."

  • @pyrotechnick420
    @pyrotechnick420 8 місяців тому +1

    I bet those 6 people who were able to sign up on day 1 were only able to do so bc they were the first 6 people to submit their applications before the system crashed

  • @-cheshire-cat
    @-cheshire-cat Рік тому +1

    Fire makes every presentation better, especially when you combine fire and the government.

  • @oliverford5367
    @oliverford5367 11 місяців тому +2

    As Joel Spolsky said, non-technical people managing software projects is like a non-surfer relying on their advisors standing on the shore to tell them what to do.
    Project managers are necessary to make decisions, but they need to know the domain. Linus Torvalds could be an ordinary dev, but he focuses on managing the whole Linux kernel. His job is to make the technical decisions and give the implementation to trusted, competent people. But an MBA who wasn't a programmer wouldn't understand the decisions that need to be made, so would come unstuck unless someone technical is really in charge.

  • @LeTtRrZ
    @LeTtRrZ 2 місяці тому

    The first half or so of this video is the story of my entire QA career.

  • @xcloudx01alt
    @xcloudx01alt 3 місяці тому

    "We'll fix it at launch"
    > only 6 sign ups and servers are dead
    "how could this happen?!"

  • @MrCmon113
    @MrCmon113 3 місяці тому +1

    It's amazing how often shit goes completely sideways because of some arbitrary deadline someone pulled out of their ass.

  • @inf3rnalis804
    @inf3rnalis804 Рік тому +2

    Holy shit at some point why didn’t anyone think we need to rebuild the entire project with people who know what they’re doing

  • @MrNobbless
    @MrNobbless 11 місяців тому +2

    "They didn't document their code"
    Sounds like good practice to me

    • @emilyau8023
      @emilyau8023 10 місяців тому

      As a business analyst whose job is to document and obtain requirements while communicating between teams so everyone is on the same page, this horrid example of organization hurts badly.

  • @rgbmew
    @rgbmew Рік тому

    another banger video love u champ

  • @GeneralNuisance00
    @GeneralNuisance00 4 місяці тому

    God i would kill for an analysis from you on the gong show that is the Phoenix payroll system

  • @darkopz
    @darkopz 7 місяців тому

    The worst part is that they consider this a full success. Guaranteed.

  • @puzzlepuzzlepuzzle760
    @puzzlepuzzlepuzzle760 Рік тому +1

    How does this channel not have more views and subs is strange to me

  • @AraiDigital
    @AraiDigital 3 місяці тому

    Yes. It seriously was that bad.
    I remember trying to get insurance through their portal and goddamn it took me almost a full month and a half to FINALLY get started.
    And yes, I mean *started*, not *completed*.
    THAT took me TWO months and a quarter.

  • @whatthepick
    @whatthepick 11 місяців тому +1

    A work of blood sweat and tears that the A Team did singlehanded more less in a 1/16th of the time

  • @benchoflemons398
    @benchoflemons398 Рік тому +3

    Government run health insurance: