My heart goes out to you. I've been there. Finding out about essential equipment that all of a sudden "no one" knows about, but its now a problem cause it stopped working. And then the "blame game" starts. Good luck there and God Bless.
Unfortunately for my CIO I think he got most of the blame. Crap may roll down hill but blame rolls up hill. As a result we're also getting our Halon system looked at and tested. God bless!
Yeah those were used for our VOIP system and also our PACS (radiology) image storage. We're starting an upgrade project so we'll see what the next storage technology will be or if it will be in a cloud somewhere. God bless!
We have a newer Mitsubishi UPS I want to say was put in around 2015 or so. It’s tall but black with a green screen on front. Ours is down too. Batteries were bad so they were replaced but now it’s some kind of control board issue. Looks like 6 industrial fuses were ordered. However we do have a full room Eaton UPS as well so anything with dual power supplies is connected to each UPS. There were some smaller devices with only one power lead so we moved them to the Eaton leg for now. Yeah we have a generator too but your right it takes time to kick over. Hope we all get our UPS situations sorted quickly! God bless!
I guess that is why larger server farms/rooms use A/B power circuits with seperate electrical distribution so you can replace failed UPS&bypass switch without interruption to the other circuit.
Hi! You could make use of some IGMP proxying between VRFs. Maybe its the way to go, but not knowing more abou the application needs its not easy to suggest solutions :)
Great suggestion! However, I will say that even the engineer that designed our network said it was not a straighforward solution to send multicast traffic between VRFs on VOSS switches. We may not even need in the end. I'll update everyone. God bless!
I manage all of the network and server equipment for several sites. Management has been fighting me on buying backup servers, and servers for test environments. Sadly in a lot of the IT world management won't listen until the exact problem you were proactively trying to prevent happens later down the road.
Exactly. And when the problem does crop up, they ask why you didn't warn them. Nevermind that you were trying to prepare for any problem, you didn't warn them about that exact problem. Been there. God bless!
@@NetworkAdminLife That is why whenever I want to make changes or buy new equipment to proactively prevent problems I always have that conversation in email for documentation in case problems arise. Thanks for the awesome content! I hope you have a fantastic day.
Littel hint, multicast on switches is igmp. It manges which device subscripes to which multicat group. And on routers you do pim/sparse which will handel the subscription coming in via igmp and subscribes the multicast groups upstream. Different vendors call it different and how to configure it, buth that are the bones ;)
I know on our old S-Series switches it was a simple as setting up PIM but for some reason it's not as straightforward on these new Fabric switches. Even the engineer that designed our network said he'd have to think about it. God bless!
That's where we came from. A hodge-podge of single purpose EMR components but no all-inclusive system. Now we're here. We'll see if they stay with this EMR system or move onto something else. God bless!
I can not imagine what the Data Center sounded like with none of that equipment running. I can only imagine all the stress to get everything back up and running correctly and finding and resolving new issues that came up as a result. Will you have to shutdown the Data Center to install the new unit?
As I said in the video I wasn't there, the server guy handled it but I can imagine that the silence was deafening. Yes, when we do the replacement we'll have to do a complete shutdown of the data center. THIS time it will be a graceful, proper shutdown. God bless!
Last Friday we had UPS failure at work as well. Took our internet out entirely. These were being monitored by an outside service. With no warning signs, the units displayed (internal short circuit). Very strange. To their credit, they came on site and replaced unit same day. How odd though they had a team and equipment ready? Someone indeed knows. More to this than we think.
It's always fun dealing with unplanned system outage. Especially at 2am in the morning. It's actually worse when it happens middle of the day and all those users putting in support tickets. Not fun.
We have a DR data center in the County building and have a domain controller running over there along with another wireless controller. I need to move our other NAC over there as well. So DNS and LDAP wwere working just fine. God bless!
I think we know how expensive these things are, however, we're talking about a hospital, which in MY mind, is a whole lot more important than your everyday company/corporation/LLC. It's a good thing that the main hospital and PACS did not lose power. I laugh, because at our Church, our UPS went down. Unfortunately, we cannot afford a backup automatic generator. Hey, though it is God's House, we have a few more options when things go down (like going home), unlike a hospital. joe. Reply
We have backups for everything at my church. If the projector goes down, we have hymnals. If we lose power, we have a piano. Amazing how we worshipped and treated patients for thousands of years without power. But yeah, it sure makes things convenient. God bless!
Keep us updated on the ups situation. God bless
For sure. I'll try to document the replacement which is SUPPOSED to happen this coming week. We'll see. God bless!
My heart goes out to you. I've been there. Finding out about essential equipment that all of a sudden "no one" knows about, but its now a problem cause it stopped working. And then the "blame game" starts. Good luck there and God Bless.
Unfortunately for my CIO I think he got most of the blame. Crap may roll down hill but blame rolls up hill. As a result we're also getting our Halon system looked at and tested. God bless!
Yeah. Not a bad Idea. Rather be safe then sorry.
YEP! EMC tech here ! Noticed all the EMC in your videos :)
Yeah those were used for our VOIP system and also our PACS (radiology) image storage. We're starting an upgrade project so we'll see what the next storage technology will be or if it will be in a cloud somewhere. God bless!
Hi Jason, I'm glad to see you hear when I gain so much inspiration.
We have a newer Mitsubishi UPS I want to say was put in around 2015 or so. It’s tall but black with a green screen on front. Ours is down too. Batteries were bad so they were replaced but now it’s some kind of control board issue. Looks like 6 industrial fuses were ordered. However we do have a full room Eaton UPS as well so anything with dual power supplies is connected to each UPS. There were some smaller devices with only one power lead so we moved them to the Eaton leg for now.
Yeah we have a generator too but your right it takes time to kick over.
Hope we all get our UPS situations sorted quickly!
God bless!
So three UPS's died within a relatively short time span. Yours, mine, and another commenter's. Most peculiar. God bless!
It sucks when orgs don't want to shell out the cash for I.T. to properly periodically test disaster recovery procedures.
Well we're definitely going to test it in a couple of weeks. Complete data center shutdown while the new UPS is installed. God bless!
I guess that is why larger server farms/rooms use A/B power circuits with seperate electrical distribution so you can replace failed UPS&bypass switch without interruption to the other circuit.
Yes, and normally this is what we try to do. God bless!
Hi! You could make use of some IGMP proxying between VRFs. Maybe its the way to go, but not knowing more abou the application needs its not easy to suggest solutions :)
Great suggestion! However, I will say that even the engineer that designed our network said it was not a straighforward solution to send multicast traffic between VRFs on VOSS switches. We may not even need in the end. I'll update everyone. God bless!
I manage all of the network and server equipment for several sites. Management has been fighting me on buying backup servers, and servers for test environments. Sadly in a lot of the IT world management won't listen until the exact problem you were proactively trying to prevent happens later down the road.
Exactly. And when the problem does crop up, they ask why you didn't warn them. Nevermind that you were trying to prepare for any problem, you didn't warn them about that exact problem. Been there. God bless!
@@NetworkAdminLife That is why whenever I want to make changes or buy new equipment to proactively prevent problems I always have that conversation in email for documentation in case problems arise. Thanks for the awesome content! I hope you have a fantastic day.
Littel hint, multicast on switches is igmp. It manges which device subscripes to which multicat group. And on routers you do pim/sparse which will handel the subscription coming in via igmp and subscribes the multicast groups upstream. Different vendors call it different and how to configure it, buth that are the bones ;)
I know on our old S-Series switches it was a simple as setting up PIM but for some reason it's not as straightforward on these new Fabric switches. Even the engineer that designed our network said he'd have to think about it. God bless!
Thanks for the interesting vid, God Bless
Glad you enjoyed it, God bless!
Very lucky, our hospital hosts are EMR and we would have been in a bad place for sure.
That's where we came from. A hodge-podge of single purpose EMR components but no all-inclusive system. Now we're here. We'll see if they stay with this EMR system or move onto something else. God bless!
I can not imagine what the Data Center sounded like with none of that equipment running. I can only imagine all the stress to get everything back up and running correctly and finding and resolving new issues that came up as a result. Will you have to shutdown the Data Center to install the new unit?
As I said in the video I wasn't there, the server guy handled it but I can imagine that the silence was deafening. Yes, when we do the replacement we'll have to do a complete shutdown of the data center. THIS time it will be a graceful, proper shutdown. God bless!
Last Friday we had UPS failure at work as well. Took our internet out entirely. These were being monitored by an outside service. With no warning signs, the units displayed (internal short circuit). Very strange. To their credit, they came on site and replaced unit same day. How odd though they had a team and equipment ready? Someone indeed knows. More to this than we think.
Agreed. I'm red-pilled and ready. Stay frosty and God bless!
It's always fun dealing with unplanned system outage. Especially at 2am in the morning. It's actually worse when it happens middle of the day and all those users putting in support tickets. Not fun.
Yes. The avalanche of tickets that come in even though you've put out the word you know what's going on and are working on it. God bless!
Then you probably had a domain controller down in the mdf too I guess, for DNS and authentication?
We have a DR data center in the County building and have a domain controller running over there along with another wireless controller. I need to move our other NAC over there as well. So DNS and LDAP wwere working just fine. God bless!
I think we know how expensive these things are, however, we're talking about a hospital, which in MY mind, is a whole lot more important than your everyday company/corporation/LLC. It's a good thing that the main hospital and PACS did not lose power. I laugh, because at our Church, our UPS went down. Unfortunately, we cannot afford a backup automatic generator. Hey, though it is God's House, we have a few more options when things go down (like going home), unlike a hospital. joe.
Reply
We have backups for everything at my church. If the projector goes down, we have hymnals. If we lose power, we have a piano. Amazing how we worshipped and treated patients for thousands of years without power. But yeah, it sure makes things convenient. God bless!
Sounds like a COA event's lol
If that's anything like CYA, then yes. God bless!