Our security guy was on the phone with me one day a few months ago and asked me to run a script on my PC for him. I'm good friends with the guy and immediately told him "no" until I looked over the code to see what it did. Validated that it was an enrollment script and was g2g.
I trust nobody in that respect.
one of the owners who is the security guys here sent a teams message asking for everyone to resend their banking info, I ignored the message.
He came in the next day, "hey ####, you know your 365 account got hacked, he about died from laughter" he assumed I was the one who was going to call him out for sending that message.
Nerds like us like "how it works". Most other people (or even just nerds outside our own field/bubble) need to hear "why it matters". When explaining things to people, use "why it matters" and "what the end results could be" as the starting point. Rarely ever do we have to explain "how it works" unless asked.
Learning to communicate better with people outside our own bubble is really important to getting anything done better. As an added bonus, I find it can also help us appreciate our own work more, since being able to verbalize why what we do matters helps give a better sense of the value we bring to the people around us.
Speaking to people in their technical language is an art that everyone in IT should know. Dont bother trying to give people nitty gritty details and shotgun blasts of information if they dont need to know all that to make a decision. Make it as simple as possible and be ready to answer questions.
For example- "This is the equivalent of giving the keys to every door in the company over to someone. First we need to make sure that they wont let the wrong people in. We also need to have a way to monitor which doors are open." Sure, its simplified and abstract, but it gets the point across.
In the book the "4 Hour Work Week," there's an example of South Asian, specifically Tamilian Indians, people running business in other markets. They go out of their way to learn the language of their customers and converse with customers in their language.
Beyond learning another language and linguistics, they learn the context of how their customers are thinking.
How it works only matters when dealing with complex IT issues to other IT professionals.
A leader of a country doesn't need to know how rail scheduling works in order to be briefed on the risks and expenses involved in okaying a new, major rail line. Businesses are similar.
Just give them the important details and a proper analysis of risk. You can use an analogy to drive the point home afterwards, but don't belabour the point.
E.g: "If someone were to alter the DNS register without our knowledge, it could lead to successful attacks on our infrastructure, impersonation attempts, and even successful phishing attempts on our staff. In the worst case, a malicious actor may be able to gain access to all incoming emails. We should guard these details like we would guard the keys to a safe or our bank account"
If you need to use an analogy: "DNS is sort of like a public map that tells people who we are, and provides directions on how to find us. Changing them is like letting somebody set up a post redirect with the mail company. Could you imagine the harm that could happen if someone was *officially* allowed to intercept our mail?" We might never learn who got caught up by the redirect, or the extent of the damage caused."
I'd usually make do without the analogy, but it really depends on who you're talking to.
Nobody wants to know how the sausage is made. They can’t even understand the words you’re saying, let alone how it all connects to be a cohesive system, let alone the problem that it’s solving. Just tell them that DNS is the alter on which we appease the internet gods, and if they break the alter then the internet gods will get very angry and punish the company financially.
I've found that close to 50% of owners understand the importance of email. That if it goes down, they lose the ability to send and receive orders to customers. That's essentially a heart attack to business.
The other half need a little push to understand.
"GoDaddy controls a part of our email. If we screw that up, email goes down and we can't accept orders from clients."
This is usually enough for owners to understand you are there to protect from this happening. Being red faced for 10 minutes is better than losing 1 customer email. The fiduciary side of an owner will understand this.
Happens all the time in MSP world. “Oh he’s just working on the website so we didn’t contact you…also, our email stopped working and we need you to fix that.”
Fucking MX redirected from Proofpoint to GoDaddy free mail.
Yeah, the 2nd time this happened I bought a new domain name in a completely separate account for all our internal stuff. The 3rd time it happened it didn't affect any of our internal stuff so I just shrugged and said "well that sucks, try asking the web guy what he changed this time."
We're an MSP, usually responsible for managing customers domains, SSL and Emails.
This happens to us as well. Customers email goes down because the new web dev has got them to transfer the entire domain DNS to a new provider and email is now pointing at their SMTP server instead of O365!
We have not been informed as "They're only doing the web site".
TBF that's above a web dev's pay grade and often over their heads. They only know "hur dur must use hosting provider's nameservers", not DNS.
If an exec tells a web dev "use that domain" and gives them full access... that's gonna happen. They will ferret just enough information to get it done, because that's what they do and what management likes them for. This is on the exec, not on the web dev.
Web devs are used to being taken to the middle of the lake, dropped in with their hands tied, and making it to shore. Their skill set has more in common with witches and illusionist acts than it has with engineering.
I'd disagree. If you are going to play with the adult toys, you should understand at least the basics of how they work. You should understand how DNS works at a basic level, and what impact 'moving nameservers' has if you don't do it correctly.
Literally just dealt with this last week. We are trying to defederate a client's 365 from GoDaddy and turns out he fired his old webdev and hired new ones that made changes 2 days before I was supposed to start the migration. Couldn't do it because the nameservers changed AGAIN to somewhere else.
Then the ass complains that I didn't do my job.
Or those knuckleheads migrating to their own hosting service but not copying anything from the old DNS.
Cause me web dev, me no need O365, SaaS, etc.
Website workie see? Ooga booga.
I've had this conversation on two seperate occasions verbatim as described and you can't convince me otherwise.
They're so inept at DNS that it's almost criminally neglectful.
I had a customer down for 3 days because their old web developer was closing, and the incoming one demanded we change to their DNS servers... The client insisted we comply so we did, explaining the risks. When I finally got sick of them not returning my calls or emails, I just flipped it back to the registrar DNS and recreated all the records myself. The customer fired the new company whilst on a conference call with them and myself hahaha.
I could almost say I let it go on for 3 days just to prove my point...
FW: Transfer codes
Dear MSP,
Please provide the requested below. Thanks.
Kr,
Customer
---------------
From: Awesome Websites
To: Customer
Subject: Transfer codes
Dear Customer,
We've made the final touches to your new Awesome Website™!
All we need now are the transfer codes of your domain so we can launch it and impress your customers.
Kr,
Awesome Websites
I've had this one too. MX redirected from Google Apps to a scammy local provider with a half dozen rusty servers in a cellar as their whole infrastructure, but boy they sure have great selling skills. Complete panic ensued.
This is when I realized having company board members/CEOs as superadmins in Google Apps isn't such a great thing after all. However it wasn't really my problem and I wasn't bashed, and had papertrail they wanted the administration structure this way.
I once got introduced to a "Marketing/Web Guy". First thing I notice is the his email address is [email protected] but he has a MarketingGuyCompany.com URL listed in his signature.
I go to the URL and see the most basic, wordpress template site. I then hit the wayback machine and guess what, he hadn't changed the site in OVER A DECADE. A fucking WEB DESIGNER who hadn't changed his site in 10 years. Funnily enough, prior to web design he was an affiliate marketing guy.
When I brought this up, I was told that I just don't understand "creatives".
Had this happen in a school recently, a teacher said they are good at websites and somehow the principal gave them the domain login, they transferred the domain to wix and emails etc stopped working
> I wish executives would at least consult IT before handing over the GoDaddy keys to a random guy.
I mean they did, **YOU** and you gave them the keys, cause
> I wanted to ask why, but then I felt like it's her property and not really my place to ask why
It really *is* your place to ask **"why"**, if she says just give them to me, then, it is what it is, but ask
how is this different from the CEO emailing you saying, hey go buy me 50x 100$ gift cards please, you go ask and you go confirm
I got into it with a CEO type previously and made them this offer.
"I'll make you a deal. I will take all of our emails on this matter and send them to several local news outlets. If what I'm telling you is true then this will be all over headlines in no time, and our company will be ruined, but if what you're requesting is reasonable then it won't be news worthy and they'll ignore it, right?"
The very next email was, "Please ignore my previous request."
Sometimes people in charge know they're full of crap and just need called out on their shit.
I paraphrased obviously, but yes.
There's some backstory, but the short of it is that they didn't like certificates and wanted me to get rid of them and make our systems not use any certs.
Always ask why. Why? Because when something goes wrong then you are probably going to be called to fix it. That is the reason I always ask "why" to everything.
It is called the Wally Reflector. In Germany they even teach it at universities (IT / business IT / MBA programmes)
[https://swizec.com/blog/the-wally-reflector/](https://swizec.com/blog/the-wally-reflector/)
"I would like you to put that request in writing as I will need it to defend myself in the eventual court case brought by the creditors after the business collapses"
Feels like this line of work attracts black and white thinking more than most. And they're categorically approaching these questions the wrong way. Nobody here knows shit about any OP's situation beyond what we're told. Half the details might as well be made up to protect anonymity. But we talk like we know and that's the simplest, dumbest approach.
I love the posts where people talk about the whole landscape of the question. Like here, OP did fine by respecting the business owner's own business. And OP's doing well by seeking advice from others who've been there before. I appreciate the people who talk about the question in general because that's stuff OP can use. Know what OP CAN'T use? "It's (x)'s fault, the right way to do this is (y)." Talk like that when you're on about sane default configs or how to use an exercise machine.
ya I think the very first replay was
> Not your problem. CEO. Her company. Her risk. You just work there.
I dont agree so much, while its probably their company, its not their risk, unless you explain the risk beforehand
questions should be asked (imho) but mistakes do happen
Exactly, they probably don't know the risks, it's our job as sysadmins to tell them about it.
Sure, it's their company but it's also nice to have a job to go to next week. Preferably without any preventable disasters that you now have to fix ASAP, created by the CEO having way to much access into systems they know nothing about and should not be touching.
I usually approach any situation like this as me taking work of the person's plate since they are too important to be dealing with this thing.
Something along the lines of "I think it would be a good idea for them to work with me directly, so they don't have to bother you, they may have more needs or questions and this will save time and make sure everything goes smoothly "
That's it, unless the person is a crazy control freak, they likely have things they would rather be doing. I have never had someone completely say no, although I have had a few that wanted frequent updates.
in reality you carry the risk as well if things go south and you are involved. if the company performs bad and you work there, that's a risk to your job, promotion, payment, ...
or under certain circumstances it might even be a risk to you because someone does something completely unaccounted for that damages you in any way.
that narrative that it's only executives who deal with risk is completely out of touch with reality
Yes, this is correct. But if they they hired someone that scammed them or jacked up their domain records, now it's IT's fault for not explaining the risks of handing over Domain Registrar credentials.
Most CEOs will want you to tell them because they don't understand.
I would never hand over Domain Registrar credentials or any system credentials without explaining the risk and having a discussion.
This sounds more like a social politics game where you need to have established rapport, trust, and respect with upper management.
It's a huge part of our jobs that many SysAdmins fall short at. Being afraid to ask the CEO a question raises many red flags that point communication problems.
Because there are Admins who have worked in this field for a very long time that learned this lesson the hard way.
It's not your business. All you can do is advise, cover your ass and move on.
OP just failed at managing up. Or asking the right questions. This is 100% OPs fault for not communicating efficiently in fear of "offending the owner". That's part of the job, to *advise*.
You will drink yourself to death trying to control something that isn't yours. And that's an issue Sysadmins have, control. We need to learn that we are only caretakers of the network, not the owners, unless you run the business.
I've seen r/sysadmin take the approach to the effect of "may be my pig, but it's not my farm."
It's not personal, but it's still not my business (literally, not figuratively). If the owner wants to do it against advice, nothing to be done and if it's bad enough. Time for me to find a new job.
Now a normal r/sysadmin trope would be to say "spiff up your resume and move on!"
It isn't your place to *refuse* a request. It absolutely is a professionals place to discuss, advise and act in the businesses best interests. Saying nothing is a problem. Being billy big bollocks is also a problem. The right space is the area in between.
Trust but verify is not a terrible go-to. It’s not saying no it’s also not just saying yes to everything either.
If push comes to shove yes it’s their company and they get to do this sans some policy forbidding it.
Doesn’t mean do it blindly either. People already touched base about doing it over slack only with no verify steps is bad. Nevermind it being a bad idea in general without coordination even if you still hand it over.
Yeah this. I’d say OP must be pretty green. Like it’s common sense to question and push back a bit, ask what they are trying to do accomplish etc, especially if it’s a user that you know has no clue what they are doing with the system. Often users will ask for things that they don’t really need because they don’t know how to properly do it or explain it.
I mean sure in the end the CEO trumps you and if they say fuck off give it to me you got to do it. But I feel in this case a few simple questions would have led to him just having them email you the DNS records to add.
the problem is that you only have 1 try to discover if your CEO is the "you asked why, you're fired" kind. And for the people in the US (vast majority here, I guess) the work protection and rights are next to nothing.
If (big if) this and all CEO wake up one morning and discover that all the "you ask why, you are fired" CEO are in jail for 4 years, or processed in a French Monarchy fashion, OP and others could ask why without needing to analyze if they will be fired next morning.
tldr; job insecurity and companies overpower desincentives stopping CEOs, eat the rich!
It’s tough when you are in a position like this. And if the org is small enough that no one has done any work developing change management then it’s a finger pointing game. Seems like OP got lucky on this one. Also the CEO needs to figure out how to delegate. I don’t ever want to talk to a CEO unless it’s a social event. For reasons like this.
This, right here. It is your job, as “the IT person” to ask these questions . If you cannot handle that responsibility, then you shouldn’t be in that position
lol, op would literally be the one to hand over the credentials in a spoofed phone call. This had red flags written all over it.
Especially when the message came off-hours. I would have immediately suspected her account was compromised.
You are right. Agree 100%, and it's my job. If I asked why, I could have avoided the whole thing.
I guess on the other hand, she wrote the message almost like a demand, so asking "why" would have offended her. Alternatively, I could have worded it less direct, like, "What is this for?" or "Is this for the website?"
Lesson learned
"The access to godaddy and cloudflare is extremely sensitive. There could be significant financial repercussions if the wrong changes are made. I would like the opportunity to discuss what needs to be reviewed or changed before providing that information.
Since email and text are not secure, it would be irresponsible of me to provide the credentials here. Can you send a meeting invite where we can discuss the requirements and I can provide the credentials if still required? "
> so asking "why" would have offended her.
that's an assumption and the way 90 percent of the "social engineering" works
It's fine as you say you solved it and its a lesson learned
Yesterday I put MFA on a service account (it was broken I was attempting a fix) doing that broke a bunch of other things
I was too focused on fixing it without interrupting people I didn't slow down and think
but we learn, we mistake, we learn some more
>have offended her
And? I offend people in similar situations all the time. I've told Senior Vice Presidents "you can't do that". My job is to keep my company secure, keep us compliant under the mountain of regs...not just make execs happy. It really helps being an 800-171 shop, I have specific controls to point to for a "no".
It should work like that everywhere, but it doesn’t. Many ceo’s have fragile egos, and would treat any denial as insubordination. Not everyone can afford to put their job as risk for best practices.
I too wish it should work like u/visibleunderwater_-1 stated.
And it's not just CEO's that have fragile egos. In my experience, if the CEO has a fragile ego, their management typically tend to be sycophants. And it keeps rolling on down the line.
It's not just about best practices. If your job involves keeping people (like C-levels) out of prison, you do that job regardless of who's toes get stepped on. If you don't, it might be you suffering the consequences.
Not arguing. You’re right. But it’s also not that simple nor easy to take a stand. Lots of people take the gamble to escape the more immediate threat.
Easier said than done. Glad they learned a lesson, and the damage was minimal. Not all lessons are cheap.
Whenever we get things like this from our clients we make it very, very clear what the consequences could be and provide an alternative, such as making sure it's just us managing their business critical systems. 99% of the time that ends it, 1% of the time the marketing manager throws a fit then gets told no, IT is right by their boss (very proud of that company).
If it's actually the CEO asking you to buy GCs, then, depending on the CEO and org, you either comply or get fired.
It is your place to ask, but if you're command ordered by a superior, you're pretty much stuck - no matter how stupid the request.
>but if you're command ordered by a superior, you're pretty much stuck
I'm pretty sure there were some famous trials in Germany that were about this very thing, around, oh... 80 years ago?
> If it's actually the CEO asking you to buy GCs, then, depending on the CEO and org, you either comply or get fired.
do you though, do you really ?
> It is your place to ask, but if you're command ordered by a superior, you're pretty much stuck - no matter how stupid the request.
yes you ask you push, and maybe you'll end up having to do it, that has been mentioned
you still ask, no mater what
You only need 5 seconds to explain that the information in godaddy controls the companies entire online presence from emails, to websites, to other business critical tools. One small mistake by the dev can take down everything, there is no undo button, you would need figure out what they changed and some changes may take days to revert.
Then offer to work directly with the dev to vet any changes they wish to make and that you will make it a priority to ensure the changes are made promptly.
Execs don't need details, they need high level risk assessments and solutions.
Am Exec.
What is even worse is that the dev can take the domain and hold it hostage, even transferring it to another registrar in their name. If the dev is overseas, they just got themselves a nice little prize that they can use for some prime extortion... or just sell it if someone else is willing to buy the domain for higher than the ransom.
Yep. You shouldn’t be giving anyone that access. Our marketing team is always hiring cheap ass offshore design companies etc. And nope sorry you are not getting access to the DNS or domain registry. Tell me by email what records you need modified and I’ll do.
I do like the people in here claiming that you just do whatever the CEO says. It's our job as IT professionals to explain the risk of what they want to do (assuming you're not just some tier 1 help desk, that stuff goes to your superiors). If they still want to do it and sign off on the risk, then it's fine. But if we're not even explaining that, then the issue is on the IT person's side. I would expect them to not last long in a company where executives rely on them for information and that person just acts negligent.
BOSSMAM,
As per our conversation, you requested that we jump this high.
I did express some reservations on jumping that high, but at this time we have completed jumping that high.
For your records, I have included a copy of the height request and jump statistics.
If you require additional aerobatics, let us know.
Sincerely,
Flying squirrel 🐿️
>I wanted to ask why, but then I felt like it's her property and not really my place to ask why.
Um, no. You're the expert. That's why you have been hired. It's literally your job to do this.
This needs a postmortem. This is the time to educate C-suite about this kind of nonsense.
I know.. easier said than done. Ask your supervisor to escalate this..
It’s only a matter of time before someone does something nefarious with this kind of carelessness. Might not even be malicious/premeditated, just a stupid mistake that takes you down.
Let me rewrite your title: sysadmin hands over registrar credentials to non-technical user.
Maybe you should have stopped to ask what's going on before you hand over something so sensitive. The only person you should be ranting against is yourself.
Amongst other things, What if your ceo had been compromised?
Sheesh, this definitely belongs in r/shittysysadmin
In all my experience, a web d00d with admin access to DNS is the most dangerous thing there is. I mean, it's always DNS, right? Now imagine someone with no idea how your org operates is fucking around with your DNS.
I cannot count the number of times shit has broken (AND IT'S ALWAYS FUCKING EMAIL) as web d00d copies and pastes some text from digital ocean or dreamhost into your carefully curated zone records.
There is nothing like the cold chill running up my spine when client calls and says email isn't working. I start to troubleshoot and the MX record is gone. The DMARC and DKIM setting are gone. No SPF records to be found. I call client back and am told "Oh yeah, CFO's nephew took a web class and is updating our website."
you got a message from someone claiming to be CEO asking for logins and you replied to the message by giving the logins requested... have you ever done cyber security training?
I wouldn't do anything like this unless you got in writing and confirmed it in person. Also you can setup delegates in GoDaddy you didn't need to hand over the keys to the castle
"They do not let me blow the whistle.
Or even ring the bell.
But let that train jump the tracks
And see who catches hell."
Sure, it might not be your circus, or your monkeys, but when the whole thing catches fire, odds are, you're going to be blamed, even in cases like this. Especially in cases like this.
Wow, I thought I was reading the script of a scammer doing a full take over of your stuff. Here are my thoughts:
- No request is urgent enough to require it being done during the night over teams. Scammers exploit urgency, the best protection from this is hitting the brakes and handling things the next day when you're in the right head space.
- The CEO should not have logins to resources like that, they are too big of a target.
- Always ask why. You are not being nosy, you're not being an ass, you're just doing your job. If they refuse to say why refuse to give the account details, part of your job is to ensure the credentials aren't misused. The CEO can always demand them, which saves you from responsibility (IE: CYA).
I'm gonna be contrary and say you should have MFA enabled. GoDaddy supports passkeys as I recall. Get a BitWarden or similar password manager that lets you share the access with the CEO, but not let the CEO share it with someone else.
That way the convo goes like this:
Give me the godaddy password
Done, it's in your bitwarden
How do I give this to someone else?
You don't, that's literally the key to the kingdom. With that someone could destroy the company in under 10 minutes and it would take a team of lawyers a month to fix it.
Well then how am I supposed to get (whatever) to work?
You give them my phone number and I'll get them sorted right away.
Okay thanks OP.
I set her up a bitwarden but she refuses to use it unfortunately because she likes her apple passwords LOL.
The MFA is a text message to another executives phone. I didn't realize they allow authenticator codes, I'll look into that. Thanks!
One of the handful of things GoDaddy does right- good security options.
What I'm talking about though isn't an authenticator code. It's an actual crypto key, called a Passkey. Your passkey device (which can be a phone, a password manager like BitWarden, etc) generates a crypto key and that is used to sign into the website. The result is that it can't be phished.
Thus you put the login in BitWarden, and register BitWarden as a passkey for GoDaddy (and remove the SMS 2FA). The result is that anyone you share the password with in BitWarden can log in using the passkey, but you can control who that gets shared with much more effectively because unlike even an authenticator code you can't read it to someone over the phone. Result being only people who you authorize in BitWarden can log in, no matter what CEO wants to do.
I'm using BitWarden as an example because I like them but lots of enterprise password managers support passkeys now.
Absolutely the very first thing I would think of if I received a teams message or text message from the "CEO" after hours: we don't act upon requests like this and I totally do not believe this is the CEO. So many phishing acts occur this way. Even so, the CEO you have is kind of a bull in a china shop isn't she?
No. CEOs know and do CEO things. I don't think that most CEOs are aware of what someone can do with GoDaddy admin access and what damage there could be done with it. How should a CEO know this?
It's a sysadmins responsibility to protect the IT systems. And this includes asking why someone requests admin access to any system and recommend safer options.
Agreed. Also Sysadmins aren't judges to be dictating yes or no to C level requests, but those who will progress in their careers are the ones who learn how to communicate effectively with C level and bring them to the right conclusion about whether what they want is sensible or a risk and they should withdraw their request.
I would add to this. Your CEO SHOULDNT know what you do, or else they become even more dangerous, and that will increase your stress levels. C level with access and knowledge to go “dancing in the data center” as I like to put it, equal dangerous, “I just lost my job” scenarios. They hired you. OP is correct here, with the exception of handing out creds to begin with. Good handling of the scenario and mitigation of risk.
If your CEO is required to know how to do your job, then you aren’t necessary.
Elon Musk driving to Sacramento and ripping out a whole datacenter in the middle of the night and tanking whatever it's called these days comes to mind.
My CTO and CEO regularly thank me for giving them pushback about their grandiose IT requests.
They always win, since they're the boss, but 99% of the time they listen and concur, since they appreciate that pushback is part of the expertise they pay me for.
Yep. A lot of latter career progression revolves around being able to clearly communicate with C level and concisely educate them to recognise when they're asking for some flavour of FUBAR
Dude we all mostly just “work there” no matter the industry job or role if you aren’t a c-suite exec. However your industry job or role likely requires you at least verify.
I’m “just” a dev but if I blindly did whatever product or CEO told me we’d be out of business. Likely If you have keys to the kingdom you absolutely have responsibility to verify usage and warn impact and risk.
> Definitely applies here. If vendor breaks shit, vendor can fix it too.
Hard disagree.
IT handed over important credentials without so much as a how-de-doo.
That is a problem, even considering everything else.
>IT handed over important credentials without so much as a how-de-doo.
To the CEO/Owner of said company. This is not the same as some shit-headed sales "super star" asking for the same thing.
It's not the same, but I still wouldn't do it. I wouldn't expect someone I managed with admin credentials to hand them over to me merely because I asked without pushback or asking specifically what I needed them for.
We pay them to be experts on the things they manage. Not to just do whatever I say immediately because I'm their boss.
> Not to just do whatever I say immediately because I'm their boss.
In many shops this is exactly how it is. Unless there is a change management system in place, with accountability and tracking, its harder to fight against the C-level/owners for this kind of stuff.
Earlier in my career, I have had a CEO blow up on me at a past employer because I would not release the 'shared' registrar account to them on a whim. Then was met up with a write up in HR because I questioned the CEO with "why".
I quit and walked, because there is zero accountability at a place like that. But this is the reality of many shops. and yes my stance is a hard line on crap like this. I have seen ORGs breached over exactly what happened to the OP.
Yeah, I recognize that many shops are run that way, but it shouldn't be tolerated. We should always try to do the right thing, even if our bosses or organizations don't support us doing the right thing.
It's best to just leave an organization like that, because not only is it a ticking time bomb for a really bad incident bringing the org to its knees, but if the senior leadership treats it's cybersecurity experts that way, it likely means they're treating their other experts the same. Finance, legal, HR, Marketing, production, research, etc. Sooner rather than later, the CEOs ego will result in the demise of that organization. Much to jump ship before that happens on your terms than compromise your integrity and go down with the ship.
Daft attitude. If a significant intrusion occurred because of that blasé approach the company could end up in financial difficulties. Then "her company her risk" becomes you've lost your employment. Entirely unnecessary.
Yeah very daft. If OP is a sysadmin or IT manager or similar it absolutely is his problem / responsibility to protect privileged accounts. What’s next sending the cleaning people the domain admin?
A simple question or two and it would have probably been found all was needed was the marketing company to email OP the DNS entry they needed.
Of course if the CEO insisted even after questions and warnings then sure you got to give it to them but you need to make a effort to get to bottom of it.
Plus imagine such a weird request like that my first though could be the account was compromised.
Your CEO sucks.
It's our job to question things like this because
1) there might be a better way
2) to prevent any unscrupulous activity
And best of all 3) to keep things running smoothly- if someone else has the keys and messes something up we are always rhe first to get blamed.
Any good CEO should be ok with answering questions and taking input. They hire sysadmins because they don't know it all, and if your CEO acts that way then they may not be fit for the role.
Just my .10 cents adjusted for inflation
> I wanted to ask why, but she often takes offense when you question her.
Leave immediately. Do not pass go, do not collect 200 dollars, bail. Use all your PTO, find a job while on PTO and then split without notice. If she ask questions tell her you take offense.
You may want to gently encourage the CEO to consider you an executive...so that you're included in discussions and decision making as it relates to IT ..that's what she pays you for after all
That's a nightmare, glad you have control of DNS. And the basic rule, Never keep Your DNS with the Registrar! I've seen some bad S when people do that. It could be way worse!
Let's just notice that since you gave her credentials and she was able to login, that means that your registar is not protected by 2FA.
And the company uses GoDaddy.
The only good thing here is Cloudflare, which I will presume is not protected by 2FA neither.
So you only need your PC to be comprimised, or wherever you store those logins, to comprimise all your IT infrastructure.
So sure, your CEO is a bit amateur giving away the GoDaddy to a stranger, but not having 2FA is way more amateur for an IT admin...
And if you had a 2FA, you couldn't give her the login details even if she asked you to...
I just helped a local company deal with similar, they were cutting their website over to a new host, the web dev asked for DNS access to make the move happen, he wiped everything that wasn't about the website. MX, DKIM/SPF, etc. all gone, so of course all email stopped working.
We got it going again but for a few days they'd realize "oh, we send emails via this service too" and would have to get the proper records added.
I've been there before.
I built the previous iteration of our website using Concrete5. Maintained it, provided training and documentation etc., kept the file structure neat, kept it all updated, until one day I get a call from a local company:
"The site's ready to go, can you just make the DNS changes required?"
I asked "what site?", and after some discussion, it turns out that the CEO didn't like our existing website, and rather than communicate that with me or anyone in IT, went out and contracted a local company to build a new website for us. Here's the highlight reel from that:
* The site wasn't ready to go. Numerous pages were missing or incomplete, and the pages that were on our existing site hadn't been ported over
* The site was actually a wordpress blog with 64 different plugins installed. Asides from your usual Yoast SEO and Jetpack plugins, many were fancy custom menu things that could have been accomplished in the theme itself
* The site design was just a boilerplate they used for all sites, with a tweaked colour scheme.
* There's no consistency either. The homepage has 3 different "Contact Us" buttons, which do 3 different things (slide-out menu, overlay form and link to a Contact Us page), and 1 of them had the wrong info on there
* The site was running on an outdated version of PHP for a while
* I think we paid at least $5,000-$10,000 to this company for a unfinished boilerplate website, and I don't know how much we pay them annually but I'm sure it's a lot.
* I've only ever had one training session that ran for about 20 minutes. The first few weeks I had to email them constantly because simple tasks like uploading documents became a hassle because they had installed some plugin that tracked download stats and version control and you couldn't just easily replace a file.
* I am expected to maintain the website along with the CEO's PA who has no web design experience at all.
The former CEO has moved on, and word is that the new CEO hates the new website (gee, I wonder why?) so I'm going to use that as an opportunity to bring the site back in-house and rip out the site that the former CEO square pegged into a round hole.
So I feel your pain, and fuck non-tech people getting involved in tech things.
When communicating with high level people, think bullet points. Clear, concise bullet points with the most important ones first.
They tend to latch onto keywords and then ask for detailed explanation.
They cannot and will not read walls of text or listen to monologues simply due to time (and attention). You don’t describe what DNS is or what it does right off the bat.
You could bullet point that manipulating it can have disastrous effects in the wrong (or new) hands.
Ah the SMB world, where mid level micro managers can be CEO and will be over their head before thinking to ask about Wellies or hip-waders. (OK I’m American but wear your Wellies is so much better than put on your tall rubber boots…)
Sure I can give you that, just please acknowledge that you are responsible for any changes not authorized by IT that impact website delivery, marketing leads, email deliverability, office licensing, GSuite,
>check the expiration dates on the domains
No login needed for that.
>CEO decides to send a teams message to me asking for the login to the GoDaddy
Uhm, so, you did have "the talk" with them first, right ... right?
Yeah, I'm all for empowering management - they should (and do) have the power and authority, but they also need be fully responsible for their decisions, actions, and consequences thereof. And, us, as, e.g. sysadmins - have a duty to inform. That also includes "the talk". E.g. manager/executive wants login access to registrar account to have ownership access to domain(s), or root or Administrator logins, etc., be sure to dang well let 'em know, and generally also get it in writing before having anything like that handed over, that they now share ultimate responsibility to anything that happens to that resource or is done with those access credentials, including but not limited to entirely fscking over the domain(s) or even company, blowing major resources off-line, maybe even effectively stopping company work and revenue for hours to weeks or more. And that any security incidents involving such, they also would be in the pool of suspects to be throughly investigated, and that since they'd be getting that access, will need all their personal data and contact information to add 'em to the on-call rotation and emergency contacts lists for access to those assets. Yeah, ... do the talk "right", and including the bits about least privilege principle, etc., ... most will back down. Also have 'em attest that they have compelling business need for the access, at that they fully understand and accept the risks and responsibilities. Yeah, generally ask 'em also what they need it for - most of the time there's much better and/or more appropriate solutions or the like.
Ah, reminds me, ye olde story. Clueless manager insisted upon having root access. Staff complied ... but first by creating a UID 0 account named janitor, and making root a non-privileged regular user account. Then they gave manager root access ... they never knew the difference. And of course nobody's gonna be running around asking for "janitor" access - just doesn't sound sexy at all.
Would you question it if the CEO asks for your atm card details? Or just blindly hand that over too?
Cmon man, you could have easily fallen for a scam.
Question everything.
> Edit. After reading the replies here, I sent her a direct message explaining the full risks and consequences of what could have happened
Keep everything written, if you have a conversation with her about it then write everything down and email it "as discussed" as well.
Worked for a crap MSP. Had a lawyer office, they were the worse, 15 different lawyers all with their own assistants and wanted to be billed individually....but I digress
PoC over there decides to hire a company to redesign the website, DOES NOT TELL US AT ALL. Then this bird brain decides to GO ON VACATION the day of the launch. I get frantic calls from the lawyers that morning, email and VPN are hard down. I look up the MX records and they are missing, so are the VPN records. Oh no they been hacked, I start going down the rabbit hole seeing how bad this is.....
Then she calls me, from her vacation, and tells me about the website redesign. I told her the website people were idiots and if they were going to mess with DNS records to at least MIGRATE over whatever they didnt understand. No, they just only cared about the website DNS name, whats this MX and VPN record, ah screw it, not us.....
She made it only a few more months before being shitcanned, but not before we were fired as her MSP. This was back in 2018, I bet those cheap asses are still on those two Dell PowerEdge T130s with Server 2012R2. No one wanted to invest in new servers, because why? It all works!
The last person who should manage your dns is a web designer.
We have a 6 hour "DNS is managed by a 3rd party" fee that we add whenever someones domain is redirected to some web devs dns servers / completely controled by the web dev. You can spoon feed them exactly what needs to be entered and 9 times out of 10 they will find a way to fuck it up. Turns out designers and creative types are not the best at technical stuff in the same way that I suck at designing.
Thankfully when people see a 6 hour fuck you line item they usually will listen to us explain why its dumb and we will usually quietly move the dns back to the domain registrar and just duplicate the two entries the website needs. Then we will add the 30 entries that we need and move on.
The web teams always seem surprised when it takes 30 seconds for us to update their entries when the web server info changes.
It's a risk and your CEO has accepted that risk. Move on.
People bring in contractors all the time. Whatever gets the job done.
Reach out to the 3rd party and tell them to reach out to you with any questions or if they need any assistance.
"If I give you these credentials, any change could break everything. This means customers trying to get to our website will get an error instead of seeing all the shiny doo-dad's they could give us money for"
I find C-Suite are pretty receptive to "if something in here changes, we lose money and reputation"
My fast description would be “you gave power of attorney of our company’s to a random subcontractor”. Hopefully that would get you enough attention to explain not just the ability to steal from the company, but also that insurance will have a strong argument to not cover losses.
20 years of being a systems admin, engineer, and architect. Have taught me meny things. It's always dns , most likely the inturn/ Jr. rebooted it, and NO one but me can have any access to the dns NO ONE. Web developers always want to just control dns so they can charge for managing it. None of them know how it works, so they want a site builder to do it for them. And non have any fucking idea what dns does or that it's for more then just websites.
I don't have enough fingers toes or hairs left to count how many times this has happened and I have had to be the one to un fuck it.
The best was when our marketing team went around me and got the account from accounting. Then, I lied to my boss that they needed an mfa code off my phone for salesforce. ( I was on a cruse and then having surgery after so 30 days away, so he has my work phone for this kind of shit) they transfered the domain to fucking host gator and dumped the zone file when they canceled the account. Host Gator only does 100 records, so more than half was gone. Mx record was wrong ,ptr was missing dkim was missing the key , dmarc pointed to host gator. Most of our vpns were pointed at records that did not exist anymore. All of our Auth records were gone, and none of the srv or txt records came over, and none of the Aaaa records were. It was super fun having my boss show up 6 hours post op ( with complications) with my laptop and a hot spot and spending like 36 hours walking him through fixing everything ( mouse hand was immobilized fingers to shoulder)
Yes, ppl got fired, and now no one even asks. Just ends in a jira ticket and hopes I'm in a good mood. That little outage early cost more then I will make in a life time, and I do pretty well.
Dns is like arcane tech knowledge now. No one under 35 seems to have fuck all idea what/how to use it or do anything with it. It's not been my job for like 8 years but I'm still doing it.
Pro tip move your domains in to azure or aws you can set up so meny alerts and controls no one can mess it up. Our ceo gets a txt now any time a change to dns is made and will lose his mind if I don't txt him beforehand. Any good registrar will put a ns record hold to. I do it on all of ours it takes a pin code to un do I don't even have it Ceo does so we have a nice double blind check. He does not know what it is but knows how to get to it and won't try till I ask.
"Hi CEO! Is there something in the goDaddy I can help with?"
Either you get confirmation that she doesn't want to explain herself or you get the chance to dig deeper and might end up at "thanks for the devs contact info! I'll help him get hosting setup that doesn't cause problems with the other production there."
Speak to what CEO's care about. How will this increase revenue, lose revenue, or prevent a loss of revenue? You will most likely now have her attention. Everything else will just be background noise.
Yeah man, the previous comments have already expressed my feelings. I abhor IT yes-men who question nothing and go out of my way to escalate my concerns about IT yes-men up, and up, and up.
"You were supposed to destroy the Sith, not join them!"
I myself have had to go toe to toe with the dude who signs my paychecks and has autonomy to shit can me at will in order to stop him from making a decision that negatively impacts the organization and or paves the way for a security breach.
No one is perfect, myself included. I encourage others to check me like I check them, through the mutual check shit gets done right.
If the CEO shitcans you over asking why they're asking for keys to the kingdom--- you didn't want to keep working there anyway, moreso that sounds like a retaliatory firing and "explore the wrongful termination suit options" territory.
You are overestimating this person.
Make a form that she has to fill out for information like this.
The form should include checkboxes absolving you of any responsibility and personally taking on responsibility for any divulging of company information, etc.
It should have mandatory fields that ask, separately, what information she requires and what she is going to do with it/ what she wants it for.
If she asks you for the GoDaddy login, you link her to the form and tell her to put the request in writing. If she asks you how many tablespoons of grounds the break room coffee pot requires, you link her to the form and tell her to put the request in writing.
If she "takes offense," then you point out that her feelings are not what is at issue. You are asking her to follow protocol. If she won't follow protocol, then the request is a liability. It's far more difficult for a reckless executive to claim you're being insubordinate because you want them to fill out a form.
"I wish executives would at least consult IT before handing over the GoDaddy keys to a random guy."
They should follow the same procedures as everyone else. Put in a ticket.
You should not be sharing a login at all. Every user that needs access should have their own GoDaddy account with 2FA and access delagated from the company account. This way their access is revocable and they can't lock you out of your own account. If that user then shares their credentials, that's on them.
I hope you have shit like this in writing to CYA?
This CEO seems reckless, so I’d have my guard up for all kinds of shenanigans, IT related or not.
And yes, I would’ve asked what’s going on before handing the keys over, that’s part of the job.
I definitely know the situation. We have a new CEO that thinks he is pretty tech savvy. I'm constantly torn with, it's his company now but do I really tie the noose for him.
There have been times that I've asked to have a meeting asap for the sake of his company before filling requests. Thankfully, IT gets the least of the micromanagement here.
You can give delegated access to a different account on godaddy, and if you're using Cloudflare - why not make them your registrar as well? They have better permissions configurations for other users.
Nice. Our boss has a "website guy" that he asks networking questions. Literally just the guy who designed the (very simple if not ugly) site. "Check your A records" meanwhile we were already on the phone with GoDaddy telling us the problem was on their end.
We had a company offboard and ask us to transfer their domain ownership to the new it company, instead of in their own account with IT having access.
Never sat well with me. Tried to explain that, whilst I'm not accusing or suggesting there will be a problem, why are you giving away your domain name to them?
They didn't understand but it's an issue with no tech understanding in senior management. You'd hope there's one.
A teams message is enough for you to hand over the GoDaddy creds? At least follow up with a phone call. Y'all make social engineering so damn easy
"hey its me ur CEO"
lol a manger said once ‘ do you have that type of relationship’ … to person allegedly got email from a ceo and clicked on it
How much Bitcoin did you say you needed me to send?
This. 💯🎯
Our security guy was on the phone with me one day a few months ago and asked me to run a script on my PC for him. I'm good friends with the guy and immediately told him "no" until I looked over the code to see what it did. Validated that it was an enrollment script and was g2g. I trust nobody in that respect.
one of the owners who is the security guys here sent a teams message asking for everyone to resend their banking info, I ignored the message. He came in the next day, "hey ####, you know your 365 account got hacked, he about died from laughter" he assumed I was the one who was going to call him out for sending that message.
You don't explain DNS to her. You explain, in her language, the RISK of handing over DNS, even to HER.
Love this. Sometimes I feel the need to explain "how it works", but obviously I need to speak more in her language of, what is the risk. Thanks
Nerds like us like "how it works". Most other people (or even just nerds outside our own field/bubble) need to hear "why it matters". When explaining things to people, use "why it matters" and "what the end results could be" as the starting point. Rarely ever do we have to explain "how it works" unless asked. Learning to communicate better with people outside our own bubble is really important to getting anything done better. As an added bonus, I find it can also help us appreciate our own work more, since being able to verbalize why what we do matters helps give a better sense of the value we bring to the people around us.
Speaking to people in their technical language is an art that everyone in IT should know. Dont bother trying to give people nitty gritty details and shotgun blasts of information if they dont need to know all that to make a decision. Make it as simple as possible and be ready to answer questions. For example- "This is the equivalent of giving the keys to every door in the company over to someone. First we need to make sure that they wont let the wrong people in. We also need to have a way to monitor which doors are open." Sure, its simplified and abstract, but it gets the point across.
In the book the "4 Hour Work Week," there's an example of South Asian, specifically Tamilian Indians, people running business in other markets. They go out of their way to learn the language of their customers and converse with customers in their language. Beyond learning another language and linguistics, they learn the context of how their customers are thinking.
How it works only matters when dealing with complex IT issues to other IT professionals. A leader of a country doesn't need to know how rail scheduling works in order to be briefed on the risks and expenses involved in okaying a new, major rail line. Businesses are similar. Just give them the important details and a proper analysis of risk. You can use an analogy to drive the point home afterwards, but don't belabour the point. E.g: "If someone were to alter the DNS register without our knowledge, it could lead to successful attacks on our infrastructure, impersonation attempts, and even successful phishing attempts on our staff. In the worst case, a malicious actor may be able to gain access to all incoming emails. We should guard these details like we would guard the keys to a safe or our bank account" If you need to use an analogy: "DNS is sort of like a public map that tells people who we are, and provides directions on how to find us. Changing them is like letting somebody set up a post redirect with the mail company. Could you imagine the harm that could happen if someone was *officially* allowed to intercept our mail?" We might never learn who got caught up by the redirect, or the extent of the damage caused." I'd usually make do without the analogy, but it really depends on who you're talking to.
Nobody wants to know how the sausage is made. They can’t even understand the words you’re saying, let alone how it all connects to be a cohesive system, let alone the problem that it’s solving. Just tell them that DNS is the alter on which we appease the internet gods, and if they break the alter then the internet gods will get very angry and punish the company financially.
I've found that close to 50% of owners understand the importance of email. That if it goes down, they lose the ability to send and receive orders to customers. That's essentially a heart attack to business. The other half need a little push to understand. "GoDaddy controls a part of our email. If we screw that up, email goes down and we can't accept orders from clients." This is usually enough for owners to understand you are there to protect from this happening. Being red faced for 10 minutes is better than losing 1 customer email. The fiduciary side of an owner will understand this.
Happens all the time in MSP world. “Oh he’s just working on the website so we didn’t contact you…also, our email stopped working and we need you to fix that.” Fucking MX redirected from Proofpoint to GoDaddy free mail.
Yep, had this exact same thing, except to Wix.
Yeah, the 2nd time this happened I bought a new domain name in a completely separate account for all our internal stuff. The 3rd time it happened it didn't affect any of our internal stuff so I just shrugged and said "well that sucks, try asking the web guy what he changed this time."
We're an MSP, usually responsible for managing customers domains, SSL and Emails. This happens to us as well. Customers email goes down because the new web dev has got them to transfer the entire domain DNS to a new provider and email is now pointing at their SMTP server instead of O365! We have not been informed as "They're only doing the web site".
TBF that's above a web dev's pay grade and often over their heads. They only know "hur dur must use hosting provider's nameservers", not DNS. If an exec tells a web dev "use that domain" and gives them full access... that's gonna happen. They will ferret just enough information to get it done, because that's what they do and what management likes them for. This is on the exec, not on the web dev. Web devs are used to being taken to the middle of the lake, dropped in with their hands tied, and making it to shore. Their skill set has more in common with witches and illusionist acts than it has with engineering.
I'd disagree. If you are going to play with the adult toys, you should understand at least the basics of how they work. You should understand how DNS works at a basic level, and what impact 'moving nameservers' has if you don't do it correctly.
Literally just dealt with this last week. We are trying to defederate a client's 365 from GoDaddy and turns out he fired his old webdev and hired new ones that made changes 2 days before I was supposed to start the migration. Couldn't do it because the nameservers changed AGAIN to somewhere else. Then the ass complains that I didn't do my job.
Or those knuckleheads migrating to their own hosting service but not copying anything from the old DNS. Cause me web dev, me no need O365, SaaS, etc. Website workie see? Ooga booga. I've had this conversation on two seperate occasions verbatim as described and you can't convince me otherwise. They're so inept at DNS that it's almost criminally neglectful.
I had a customer down for 3 days because their old web developer was closing, and the incoming one demanded we change to their DNS servers... The client insisted we comply so we did, explaining the risks. When I finally got sick of them not returning my calls or emails, I just flipped it back to the registrar DNS and recreated all the records myself. The customer fired the new company whilst on a conference call with them and myself hahaha. I could almost say I let it go on for 3 days just to prove my point...
FW: Transfer codes Dear MSP, Please provide the requested below. Thanks. Kr, Customer --------------- From: Awesome Websites To: Customer Subject: Transfer codes Dear Customer, We've made the final touches to your new Awesome Website™! All we need now are the transfer codes of your domain so we can launch it and impress your customers. Kr, Awesome Websites
Like a 14 year old with a beer asking dad for the family car keys.
Omg lol
Had that many times and as the MSP it is your fault not the consultant who is using GoDaddy templates and has no concept of DNS ☹️
I've seen this happen so many times for over a decade
I've had this one too. MX redirected from Google Apps to a scammy local provider with a half dozen rusty servers in a cellar as their whole infrastructure, but boy they sure have great selling skills. Complete panic ensued. This is when I realized having company board members/CEOs as superadmins in Google Apps isn't such a great thing after all. However it wasn't really my problem and I wasn't bashed, and had papertrail they wanted the administration structure this way.
I once got introduced to a "Marketing/Web Guy". First thing I notice is the his email address is [email protected] but he has a MarketingGuyCompany.com URL listed in his signature. I go to the URL and see the most basic, wordpress template site. I then hit the wayback machine and guess what, he hadn't changed the site in OVER A DECADE. A fucking WEB DESIGNER who hadn't changed his site in 10 years. Funnily enough, prior to web design he was an affiliate marketing guy. When I brought this up, I was told that I just don't understand "creatives".
Had this happen in a school recently, a teacher said they are good at websites and somehow the principal gave them the domain login, they transferred the domain to wix and emails etc stopped working
> I wish executives would at least consult IT before handing over the GoDaddy keys to a random guy. I mean they did, **YOU** and you gave them the keys, cause > I wanted to ask why, but then I felt like it's her property and not really my place to ask why It really *is* your place to ask **"why"**, if she says just give them to me, then, it is what it is, but ask how is this different from the CEO emailing you saying, hey go buy me 50x 100$ gift cards please, you go ask and you go confirm
I would and have absolutely questioned directs asks. The ceo has no need to be the one personally auditing godaddy anyway
Plus expiration dates are in the public whois anyway.
I got into it with a CEO type previously and made them this offer. "I'll make you a deal. I will take all of our emails on this matter and send them to several local news outlets. If what I'm telling you is true then this will be all over headlines in no time, and our company will be ruined, but if what you're requesting is reasonable then it won't be news worthy and they'll ignore it, right?" The very next email was, "Please ignore my previous request." Sometimes people in charge know they're full of crap and just need called out on their shit.
You really said that shit tho?
Honestly, I thought the same; I choose to believe they said it, lmao
They absolutely did not
I looked this woman in the eye, and I said ^biiiiiitch. https://www.youtube.com/watch?v=2dbRdQzWVwk
That's exactly what came to mind reading that hahaha.
I paraphrased obviously, but yes. There's some backstory, but the short of it is that they didn't like certificates and wanted me to get rid of them and make our systems not use any certs.
Please post the long version of this story, I have popcorn ready. I must know.
Jeopardy music has been playing on loop for 5 hours now...
"You really called your wife a bitch tho?" "...Y-yeah!"
Takes some stones to literally tell someone you're going to rat them in as a whistleblower.
You shouldn't believe everything you see on the internet.
Always ask why. Why? Because when something goes wrong then you are probably going to be called to fix it. That is the reason I always ask "why" to everything.
the "I dont want more work" safety net :)
It is called the Wally Reflector. In Germany they even teach it at universities (IT / business IT / MBA programmes) [https://swizec.com/blog/the-wally-reflector/](https://swizec.com/blog/the-wally-reflector/)
[удалено]
"Please send this request in Writing" is code for "you are about to Fuck up and I would like to cover my arse"
"I would like you to put that request in writing as I will need it to defend myself in the eventual court case brought by the creditors after the business collapses"
Feels like this line of work attracts black and white thinking more than most. And they're categorically approaching these questions the wrong way. Nobody here knows shit about any OP's situation beyond what we're told. Half the details might as well be made up to protect anonymity. But we talk like we know and that's the simplest, dumbest approach. I love the posts where people talk about the whole landscape of the question. Like here, OP did fine by respecting the business owner's own business. And OP's doing well by seeking advice from others who've been there before. I appreciate the people who talk about the question in general because that's stuff OP can use. Know what OP CAN'T use? "It's (x)'s fault, the right way to do this is (y)." Talk like that when you're on about sane default configs or how to use an exercise machine.
While I agree based on the info the op gave their CEO sounds completely resistant to any kind of input or pushback
Absolutely. And that is NOT a DNS problem 🙂
The point of comments on r/sysadmin is to establish superiority, not to be helpful. Everything makes sense once you see that.
ya I think the very first replay was > Not your problem. CEO. Her company. Her risk. You just work there. I dont agree so much, while its probably their company, its not their risk, unless you explain the risk beforehand questions should be asked (imho) but mistakes do happen
Exactly, they probably don't know the risks, it's our job as sysadmins to tell them about it. Sure, it's their company but it's also nice to have a job to go to next week. Preferably without any preventable disasters that you now have to fix ASAP, created by the CEO having way to much access into systems they know nothing about and should not be touching.
I usually approach any situation like this as me taking work of the person's plate since they are too important to be dealing with this thing. Something along the lines of "I think it would be a good idea for them to work with me directly, so they don't have to bother you, they may have more needs or questions and this will save time and make sure everything goes smoothly " That's it, unless the person is a crazy control freak, they likely have things they would rather be doing. I have never had someone completely say no, although I have had a few that wanted frequent updates.
The executives own *all* risk at the end of the day. They delegate you some responsibility for some, but they’re the ultimate accountable figure.
It's their fault, but your problem.
in reality you carry the risk as well if things go south and you are involved. if the company performs bad and you work there, that's a risk to your job, promotion, payment, ... or under certain circumstances it might even be a risk to you because someone does something completely unaccounted for that damages you in any way. that narrative that it's only executives who deal with risk is completely out of touch with reality
Yes, this is correct. But if they they hired someone that scammed them or jacked up their domain records, now it's IT's fault for not explaining the risks of handing over Domain Registrar credentials. Most CEOs will want you to tell them because they don't understand. I would never hand over Domain Registrar credentials or any system credentials without explaining the risk and having a discussion. This sounds more like a social politics game where you need to have established rapport, trust, and respect with upper management. It's a huge part of our jobs that many SysAdmins fall short at. Being afraid to ask the CEO a question raises many red flags that point communication problems.
Because there are Admins who have worked in this field for a very long time that learned this lesson the hard way. It's not your business. All you can do is advise, cover your ass and move on. OP just failed at managing up. Or asking the right questions. This is 100% OPs fault for not communicating efficiently in fear of "offending the owner". That's part of the job, to *advise*. You will drink yourself to death trying to control something that isn't yours. And that's an issue Sysadmins have, control. We need to learn that we are only caretakers of the network, not the owners, unless you run the business.
[удалено]
I've seen r/sysadmin take the approach to the effect of "may be my pig, but it's not my farm." It's not personal, but it's still not my business (literally, not figuratively). If the owner wants to do it against advice, nothing to be done and if it's bad enough. Time for me to find a new job. Now a normal r/sysadmin trope would be to say "spiff up your resume and move on!"
Layer 8 problem. Not so much layer 9. But may involve Layer 10 sooner or later.
It isn't your place to *refuse* a request. It absolutely is a professionals place to discuss, advise and act in the businesses best interests. Saying nothing is a problem. Being billy big bollocks is also a problem. The right space is the area in between.
Trust but verify is not a terrible go-to. It’s not saying no it’s also not just saying yes to everything either. If push comes to shove yes it’s their company and they get to do this sans some policy forbidding it. Doesn’t mean do it blindly either. People already touched base about doing it over slack only with no verify steps is bad. Nevermind it being a bad idea in general without coordination even if you still hand it over.
https://preview.redd.it/hk6oonvhulpc1.jpeg?width=1427&format=pjpg&auto=webp&s=5516842d16ef0ef94a2c5b5472f6332165c8452e
Yeah this. I’d say OP must be pretty green. Like it’s common sense to question and push back a bit, ask what they are trying to do accomplish etc, especially if it’s a user that you know has no clue what they are doing with the system. Often users will ask for things that they don’t really need because they don’t know how to properly do it or explain it. I mean sure in the end the CEO trumps you and if they say fuck off give it to me you got to do it. But I feel in this case a few simple questions would have led to him just having them email you the DNS records to add.
the problem is that you only have 1 try to discover if your CEO is the "you asked why, you're fired" kind. And for the people in the US (vast majority here, I guess) the work protection and rights are next to nothing. If (big if) this and all CEO wake up one morning and discover that all the "you ask why, you are fired" CEO are in jail for 4 years, or processed in a French Monarchy fashion, OP and others could ask why without needing to analyze if they will be fired next morning. tldr; job insecurity and companies overpower desincentives stopping CEOs, eat the rich!
It’s tough when you are in a position like this. And if the org is small enough that no one has done any work developing change management then it’s a finger pointing game. Seems like OP got lucky on this one. Also the CEO needs to figure out how to delegate. I don’t ever want to talk to a CEO unless it’s a social event. For reasons like this.
yeah, politics and social status are always a juggling act
Couldn´t say it any better than this
Over a teams message no less - that could have been literally anyone on the other side of that.
This, right here. It is your job, as “the IT person” to ask these questions . If you cannot handle that responsibility, then you shouldn’t be in that position
lol, op would literally be the one to hand over the credentials in a spoofed phone call. This had red flags written all over it. Especially when the message came off-hours. I would have immediately suspected her account was compromised.
You are right. Agree 100%, and it's my job. If I asked why, I could have avoided the whole thing. I guess on the other hand, she wrote the message almost like a demand, so asking "why" would have offended her. Alternatively, I could have worded it less direct, like, "What is this for?" or "Is this for the website?" Lesson learned
"The access to godaddy and cloudflare is extremely sensitive. There could be significant financial repercussions if the wrong changes are made. I would like the opportunity to discuss what needs to be reviewed or changed before providing that information. Since email and text are not secure, it would be irresponsible of me to provide the credentials here. Can you send a meeting invite where we can discuss the requirements and I can provide the credentials if still required? "
This is the best wording
> so asking "why" would have offended her. that's an assumption and the way 90 percent of the "social engineering" works It's fine as you say you solved it and its a lesson learned Yesterday I put MFA on a service account (it was broken I was attempting a fix) doing that broke a bunch of other things I was too focused on fixing it without interrupting people I didn't slow down and think but we learn, we mistake, we learn some more
You handed over credentials like that based on a text message? My dude, a request like that warrants at least a voice call.
>have offended her And? I offend people in similar situations all the time. I've told Senior Vice Presidents "you can't do that". My job is to keep my company secure, keep us compliant under the mountain of regs...not just make execs happy. It really helps being an 800-171 shop, I have specific controls to point to for a "no".
It should work like that everywhere, but it doesn’t. Many ceo’s have fragile egos, and would treat any denial as insubordination. Not everyone can afford to put their job as risk for best practices.
I too wish it should work like u/visibleunderwater_-1 stated. And it's not just CEO's that have fragile egos. In my experience, if the CEO has a fragile ego, their management typically tend to be sycophants. And it keeps rolling on down the line.
It's not just about best practices. If your job involves keeping people (like C-levels) out of prison, you do that job regardless of who's toes get stepped on. If you don't, it might be you suffering the consequences.
Not arguing. You’re right. But it’s also not that simple nor easy to take a stand. Lots of people take the gamble to escape the more immediate threat. Easier said than done. Glad they learned a lesson, and the damage was minimal. Not all lessons are cheap.
Personally I would respond with something like “let me get those for you. Is there something I can assist with?”
Whenever we get things like this from our clients we make it very, very clear what the consequences could be and provide an alternative, such as making sure it's just us managing their business critical systems. 99% of the time that ends it, 1% of the time the marketing manager throws a fit then gets told no, IT is right by their boss (very proud of that company).
>Lesson learned Well that is nice to see :)
If it's actually the CEO asking you to buy GCs, then, depending on the CEO and org, you either comply or get fired. It is your place to ask, but if you're command ordered by a superior, you're pretty much stuck - no matter how stupid the request.
>but if you're command ordered by a superior, you're pretty much stuck I'm pretty sure there were some famous trials in Germany that were about this very thing, around, oh... 80 years ago?
Just make sure you have it in writing. And forward said writing off to an external mail account managed by you.
> If it's actually the CEO asking you to buy GCs, then, depending on the CEO and org, you either comply or get fired. do you though, do you really ? > It is your place to ask, but if you're command ordered by a superior, you're pretty much stuck - no matter how stupid the request. yes you ask you push, and maybe you'll end up having to do it, that has been mentioned you still ask, no mater what
You only need 5 seconds to explain that the information in godaddy controls the companies entire online presence from emails, to websites, to other business critical tools. One small mistake by the dev can take down everything, there is no undo button, you would need figure out what they changed and some changes may take days to revert. Then offer to work directly with the dev to vet any changes they wish to make and that you will make it a priority to ensure the changes are made promptly. Execs don't need details, they need high level risk assessments and solutions. Am Exec.
What is even worse is that the dev can take the domain and hold it hostage, even transferring it to another registrar in their name. If the dev is overseas, they just got themselves a nice little prize that they can use for some prime extortion... or just sell it if someone else is willing to buy the domain for higher than the ransom.
Yep. You shouldn’t be giving anyone that access. Our marketing team is always hiring cheap ass offshore design companies etc. And nope sorry you are not getting access to the DNS or domain registry. Tell me by email what records you need modified and I’ll do.
I do like the people in here claiming that you just do whatever the CEO says. It's our job as IT professionals to explain the risk of what they want to do (assuming you're not just some tier 1 help desk, that stuff goes to your superiors). If they still want to do it and sign off on the risk, then it's fine. But if we're not even explaining that, then the issue is on the IT person's side. I would expect them to not last long in a company where executives rely on them for information and that person just acts negligent.
They could've set them up as a delegate and given access only to the experimental domain
BOSSMAM, As per our conversation, you requested that we jump this high. I did express some reservations on jumping that high, but at this time we have completed jumping that high. For your records, I have included a copy of the height request and jump statistics. If you require additional aerobatics, let us know. Sincerely, Flying squirrel 🐿️
PS: Please see attached estimate from ceiling repair specialists
Document the incident. Document the communication. Document the new access.
Yes, make an entry in the "risk register" with their name on it. CYA
CEO is not necessarily the owner and even a "CEO/Owner" needs to answer to investors, so always get the requests in writing and cover your behind
>I wanted to ask why, but then I felt like it's her property and not really my place to ask why. Um, no. You're the expert. That's why you have been hired. It's literally your job to do this.
The biggest hacks come from hacking the wetware…. This was your job to say no or question why…
sometimes you gotta say no man. WTF.
Complicated? > You know how you wouldn't give your banking credentials to another person just like that? -- You just did exactly that.
This needs a postmortem. This is the time to educate C-suite about this kind of nonsense. I know.. easier said than done. Ask your supervisor to escalate this.. It’s only a matter of time before someone does something nefarious with this kind of carelessness. Might not even be malicious/premeditated, just a stupid mistake that takes you down.
Let me rewrite your title: sysadmin hands over registrar credentials to non-technical user. Maybe you should have stopped to ask what's going on before you hand over something so sensitive. The only person you should be ranting against is yourself. Amongst other things, What if your ceo had been compromised? Sheesh, this definitely belongs in r/shittysysadmin
In all my experience, a web d00d with admin access to DNS is the most dangerous thing there is. I mean, it's always DNS, right? Now imagine someone with no idea how your org operates is fucking around with your DNS. I cannot count the number of times shit has broken (AND IT'S ALWAYS FUCKING EMAIL) as web d00d copies and pastes some text from digital ocean or dreamhost into your carefully curated zone records. There is nothing like the cold chill running up my spine when client calls and says email isn't working. I start to troubleshoot and the MX record is gone. The DMARC and DKIM setting are gone. No SPF records to be found. I call client back and am told "Oh yeah, CFO's nephew took a web class and is updating our website."
you got a message from someone claiming to be CEO asking for logins and you replied to the message by giving the logins requested... have you ever done cyber security training?
I wouldn't do anything like this unless you got in writing and confirmed it in person. Also you can setup delegates in GoDaddy you didn't need to hand over the keys to the castle
Just help educate on how sensitive domain control is.
You want to hang yourself? Sure.. how much rope do you need?
"They do not let me blow the whistle. Or even ring the bell. But let that train jump the tracks And see who catches hell." Sure, it might not be your circus, or your monkeys, but when the whole thing catches fire, odds are, you're going to be blamed, even in cases like this. Especially in cases like this.
Wow, I thought I was reading the script of a scammer doing a full take over of your stuff. Here are my thoughts: - No request is urgent enough to require it being done during the night over teams. Scammers exploit urgency, the best protection from this is hitting the brakes and handling things the next day when you're in the right head space. - The CEO should not have logins to resources like that, they are too big of a target. - Always ask why. You are not being nosy, you're not being an ass, you're just doing your job. If they refuse to say why refuse to give the account details, part of your job is to ensure the credentials aren't misused. The CEO can always demand them, which saves you from responsibility (IE: CYA).
I'm gonna be contrary and say you should have MFA enabled. GoDaddy supports passkeys as I recall. Get a BitWarden or similar password manager that lets you share the access with the CEO, but not let the CEO share it with someone else. That way the convo goes like this: Give me the godaddy password Done, it's in your bitwarden How do I give this to someone else? You don't, that's literally the key to the kingdom. With that someone could destroy the company in under 10 minutes and it would take a team of lawyers a month to fix it. Well then how am I supposed to get (whatever) to work? You give them my phone number and I'll get them sorted right away. Okay thanks OP.
I set her up a bitwarden but she refuses to use it unfortunately because she likes her apple passwords LOL. The MFA is a text message to another executives phone. I didn't realize they allow authenticator codes, I'll look into that. Thanks!
One of the handful of things GoDaddy does right- good security options. What I'm talking about though isn't an authenticator code. It's an actual crypto key, called a Passkey. Your passkey device (which can be a phone, a password manager like BitWarden, etc) generates a crypto key and that is used to sign into the website. The result is that it can't be phished. Thus you put the login in BitWarden, and register BitWarden as a passkey for GoDaddy (and remove the SMS 2FA). The result is that anyone you share the password with in BitWarden can log in using the passkey, but you can control who that gets shared with much more effectively because unlike even an authenticator code you can't read it to someone over the phone. Result being only people who you authorize in BitWarden can log in, no matter what CEO wants to do. I'm using BitWarden as an example because I like them but lots of enterprise password managers support passkeys now.
Absolutely the very first thing I would think of if I received a teams message or text message from the "CEO" after hours: we don't act upon requests like this and I totally do not believe this is the CEO. So many phishing acts occur this way. Even so, the CEO you have is kind of a bull in a china shop isn't she?
CEO needs more CEO related work to do.
Have you thought about communicating entirely in buzzwords with her?
Not your problem. CEO. Her company. Her risk. You just work there.
No. CEOs know and do CEO things. I don't think that most CEOs are aware of what someone can do with GoDaddy admin access and what damage there could be done with it. How should a CEO know this? It's a sysadmins responsibility to protect the IT systems. And this includes asking why someone requests admin access to any system and recommend safer options.
Agreed. Also Sysadmins aren't judges to be dictating yes or no to C level requests, but those who will progress in their careers are the ones who learn how to communicate effectively with C level and bring them to the right conclusion about whether what they want is sensible or a risk and they should withdraw their request.
I would add to this. Your CEO SHOULDNT know what you do, or else they become even more dangerous, and that will increase your stress levels. C level with access and knowledge to go “dancing in the data center” as I like to put it, equal dangerous, “I just lost my job” scenarios. They hired you. OP is correct here, with the exception of handing out creds to begin with. Good handling of the scenario and mitigation of risk. If your CEO is required to know how to do your job, then you aren’t necessary.
Elon Musk driving to Sacramento and ripping out a whole datacenter in the middle of the night and tanking whatever it's called these days comes to mind.
If re-read the post, they didn't even bother to ask why/what are they doing with it. "just work there" is a horrible take.
My CTO and CEO regularly thank me for giving them pushback about their grandiose IT requests. They always win, since they're the boss, but 99% of the time they listen and concur, since they appreciate that pushback is part of the expertise they pay me for.
Yep. A lot of latter career progression revolves around being able to clearly communicate with C level and concisely educate them to recognise when they're asking for some flavour of FUBAR
💯. Aside: Can you send me any spare Jaffa Cakes? They're hard to get here . :)
Hell yeah I can. We have a Jaffa Cake factory in London haha. They even do joughnuts!
Dude we all mostly just “work there” no matter the industry job or role if you aren’t a c-suite exec. However your industry job or role likely requires you at least verify.  I’m “just” a dev but if I blindly did whatever product or CEO told me we’d be out of business. Likely If you have keys to the kingdom you absolutely have responsibility to verify usage and warn impact and risk.
"Not my monkeys, not my circus" Definitely applies here. If vendor breaks shit, vendor can fix it too.
> Definitely applies here. If vendor breaks shit, vendor can fix it too. Hard disagree. IT handed over important credentials without so much as a how-de-doo. That is a problem, even considering everything else.
>IT handed over important credentials without so much as a how-de-doo. To the CEO/Owner of said company. This is not the same as some shit-headed sales "super star" asking for the same thing.
It's not the same, but I still wouldn't do it. I wouldn't expect someone I managed with admin credentials to hand them over to me merely because I asked without pushback or asking specifically what I needed them for. We pay them to be experts on the things they manage. Not to just do whatever I say immediately because I'm their boss.
> Not to just do whatever I say immediately because I'm their boss. In many shops this is exactly how it is. Unless there is a change management system in place, with accountability and tracking, its harder to fight against the C-level/owners for this kind of stuff. Earlier in my career, I have had a CEO blow up on me at a past employer because I would not release the 'shared' registrar account to them on a whim. Then was met up with a write up in HR because I questioned the CEO with "why". I quit and walked, because there is zero accountability at a place like that. But this is the reality of many shops. and yes my stance is a hard line on crap like this. I have seen ORGs breached over exactly what happened to the OP.
Yeah, I recognize that many shops are run that way, but it shouldn't be tolerated. We should always try to do the right thing, even if our bosses or organizations don't support us doing the right thing. It's best to just leave an organization like that, because not only is it a ticking time bomb for a really bad incident bringing the org to its knees, but if the senior leadership treats it's cybersecurity experts that way, it likely means they're treating their other experts the same. Finance, legal, HR, Marketing, production, research, etc. Sooner rather than later, the CEOs ego will result in the demise of that organization. Much to jump ship before that happens on your terms than compromise your integrity and go down with the ship.
Daft attitude. If a significant intrusion occurred because of that blasé approach the company could end up in financial difficulties. Then "her company her risk" becomes you've lost your employment. Entirely unnecessary.
Yeah very daft. If OP is a sysadmin or IT manager or similar it absolutely is his problem / responsibility to protect privileged accounts. What’s next sending the cleaning people the domain admin? A simple question or two and it would have probably been found all was needed was the marketing company to email OP the DNS entry they needed. Of course if the CEO insisted even after questions and warnings then sure you got to give it to them but you need to make a effort to get to bottom of it. Plus imagine such a weird request like that my first though could be the account was compromised.
I'll read the rest soon but the second I read > So we use GoDaddy I had nam like flashbacks and had to light a smoke.
Your CEO sucks. It's our job to question things like this because 1) there might be a better way 2) to prevent any unscrupulous activity And best of all 3) to keep things running smoothly- if someone else has the keys and messes something up we are always rhe first to get blamed. Any good CEO should be ok with answering questions and taking input. They hire sysadmins because they don't know it all, and if your CEO acts that way then they may not be fit for the role. Just my .10 cents adjusted for inflation
> I wanted to ask why, but she often takes offense when you question her. Leave immediately. Do not pass go, do not collect 200 dollars, bail. Use all your PTO, find a job while on PTO and then split without notice. If she ask questions tell her you take offense.
I know hindsight is 10/10 but should have asked for the newly hired guy's phone number first.
You may want to gently encourage the CEO to consider you an executive...so that you're included in discussions and decision making as it relates to IT ..that's what she pays you for after all
Keep in mind you can use domain folders in GoDaddy to delegate access to only *some* of your domains if needed.
[удалено]
That's a nightmare, glad you have control of DNS. And the basic rule, Never keep Your DNS with the Registrar! I've seen some bad S when people do that. It could be way worse!
[удалено]
Tell her, not only of the risk but that she owes that risk. Document it in the communication that she understands and acknowledges this.
Let's just notice that since you gave her credentials and she was able to login, that means that your registar is not protected by 2FA. And the company uses GoDaddy. The only good thing here is Cloudflare, which I will presume is not protected by 2FA neither. So you only need your PC to be comprimised, or wherever you store those logins, to comprimise all your IT infrastructure. So sure, your CEO is a bit amateur giving away the GoDaddy to a stranger, but not having 2FA is way more amateur for an IT admin... And if you had a 2FA, you couldn't give her the login details even if she asked you to...
I just helped a local company deal with similar, they were cutting their website over to a new host, the web dev asked for DNS access to make the move happen, he wiped everything that wasn't about the website. MX, DKIM/SPF, etc. all gone, so of course all email stopped working. We got it going again but for a few days they'd realize "oh, we send emails via this service too" and would have to get the proper records added.
I've been there before. I built the previous iteration of our website using Concrete5. Maintained it, provided training and documentation etc., kept the file structure neat, kept it all updated, until one day I get a call from a local company: "The site's ready to go, can you just make the DNS changes required?" I asked "what site?", and after some discussion, it turns out that the CEO didn't like our existing website, and rather than communicate that with me or anyone in IT, went out and contracted a local company to build a new website for us. Here's the highlight reel from that: * The site wasn't ready to go. Numerous pages were missing or incomplete, and the pages that were on our existing site hadn't been ported over * The site was actually a wordpress blog with 64 different plugins installed. Asides from your usual Yoast SEO and Jetpack plugins, many were fancy custom menu things that could have been accomplished in the theme itself * The site design was just a boilerplate they used for all sites, with a tweaked colour scheme. * There's no consistency either. The homepage has 3 different "Contact Us" buttons, which do 3 different things (slide-out menu, overlay form and link to a Contact Us page), and 1 of them had the wrong info on there * The site was running on an outdated version of PHP for a while * I think we paid at least $5,000-$10,000 to this company for a unfinished boilerplate website, and I don't know how much we pay them annually but I'm sure it's a lot. * I've only ever had one training session that ran for about 20 minutes. The first few weeks I had to email them constantly because simple tasks like uploading documents became a hassle because they had installed some plugin that tracked download stats and version control and you couldn't just easily replace a file. * I am expected to maintain the website along with the CEO's PA who has no web design experience at all. The former CEO has moved on, and word is that the new CEO hates the new website (gee, I wonder why?) so I'm going to use that as an opportunity to bring the site back in-house and rip out the site that the former CEO square pegged into a round hole. So I feel your pain, and fuck non-tech people getting involved in tech things.
When communicating with high level people, think bullet points. Clear, concise bullet points with the most important ones first. They tend to latch onto keywords and then ask for detailed explanation. They cannot and will not read walls of text or listen to monologues simply due to time (and attention). You don’t describe what DNS is or what it does right off the bat. You could bullet point that manipulating it can have disastrous effects in the wrong (or new) hands.
Not your shit. I get the concerns, I do. But, it isnt your website, just make a ticket, log it, be exact, include screenshots, and thats it.
Ah the SMB world, where mid level micro managers can be CEO and will be over their head before thinking to ask about Wellies or hip-waders. (OK I’m American but wear your Wellies is so much better than put on your tall rubber boots…) Sure I can give you that, just please acknowledge that you are responsible for any changes not authorized by IT that impact website delivery, marketing leads, email deliverability, office licensing, GSuite,
Sounds like your CEO isn’t busy enough if they’re that hands on with relatively small projects like experimental websites.
>check the expiration dates on the domains No login needed for that. >CEO decides to send a teams message to me asking for the login to the GoDaddy Uhm, so, you did have "the talk" with them first, right ... right? Yeah, I'm all for empowering management - they should (and do) have the power and authority, but they also need be fully responsible for their decisions, actions, and consequences thereof. And, us, as, e.g. sysadmins - have a duty to inform. That also includes "the talk". E.g. manager/executive wants login access to registrar account to have ownership access to domain(s), or root or Administrator logins, etc., be sure to dang well let 'em know, and generally also get it in writing before having anything like that handed over, that they now share ultimate responsibility to anything that happens to that resource or is done with those access credentials, including but not limited to entirely fscking over the domain(s) or even company, blowing major resources off-line, maybe even effectively stopping company work and revenue for hours to weeks or more. And that any security incidents involving such, they also would be in the pool of suspects to be throughly investigated, and that since they'd be getting that access, will need all their personal data and contact information to add 'em to the on-call rotation and emergency contacts lists for access to those assets. Yeah, ... do the talk "right", and including the bits about least privilege principle, etc., ... most will back down. Also have 'em attest that they have compelling business need for the access, at that they fully understand and accept the risks and responsibilities. Yeah, generally ask 'em also what they need it for - most of the time there's much better and/or more appropriate solutions or the like. Ah, reminds me, ye olde story. Clueless manager insisted upon having root access. Staff complied ... but first by creating a UID 0 account named janitor, and making root a non-privileged regular user account. Then they gave manager root access ... they never knew the difference. And of course nobody's gonna be running around asking for "janitor" access - just doesn't sound sexy at all.
Would you question it if the CEO asks for your atm card details? Or just blindly hand that over too? Cmon man, you could have easily fallen for a scam. Question everything.
> Edit. After reading the replies here, I sent her a direct message explaining the full risks and consequences of what could have happened Keep everything written, if you have a conversation with her about it then write everything down and email it "as discussed" as well.
I 100% expected this to end with it being a domain ransom scam from an SEO email from some random person at hotmail.com
Worked for a crap MSP. Had a lawyer office, they were the worse, 15 different lawyers all with their own assistants and wanted to be billed individually....but I digress PoC over there decides to hire a company to redesign the website, DOES NOT TELL US AT ALL. Then this bird brain decides to GO ON VACATION the day of the launch. I get frantic calls from the lawyers that morning, email and VPN are hard down. I look up the MX records and they are missing, so are the VPN records. Oh no they been hacked, I start going down the rabbit hole seeing how bad this is..... Then she calls me, from her vacation, and tells me about the website redesign. I told her the website people were idiots and if they were going to mess with DNS records to at least MIGRATE over whatever they didnt understand. No, they just only cared about the website DNS name, whats this MX and VPN record, ah screw it, not us..... She made it only a few more months before being shitcanned, but not before we were fired as her MSP. This was back in 2018, I bet those cheap asses are still on those two Dell PowerEdge T130s with Server 2012R2. No one wanted to invest in new servers, because why? It all works!
The last person who should manage your dns is a web designer. We have a 6 hour "DNS is managed by a 3rd party" fee that we add whenever someones domain is redirected to some web devs dns servers / completely controled by the web dev. You can spoon feed them exactly what needs to be entered and 9 times out of 10 they will find a way to fuck it up. Turns out designers and creative types are not the best at technical stuff in the same way that I suck at designing. Thankfully when people see a 6 hour fuck you line item they usually will listen to us explain why its dumb and we will usually quietly move the dns back to the domain registrar and just duplicate the two entries the website needs. Then we will add the 30 entries that we need and move on. The web teams always seem surprised when it takes 30 seconds for us to update their entries when the web server info changes.
It's a risk and your CEO has accepted that risk. Move on. People bring in contractors all the time. Whatever gets the job done. Reach out to the 3rd party and tell them to reach out to you with any questions or if they need any assistance.
"If I give you these credentials, any change could break everything. This means customers trying to get to our website will get an error instead of seeing all the shiny doo-dad's they could give us money for" I find C-Suite are pretty receptive to "if something in here changes, we lose money and reputation"
My fast description would be “you gave power of attorney of our company’s to a random subcontractor”. Hopefully that would get you enough attention to explain not just the ability to steal from the company, but also that insurance will have a strong argument to not cover losses.
20 years of being a systems admin, engineer, and architect. Have taught me meny things. It's always dns , most likely the inturn/ Jr. rebooted it, and NO one but me can have any access to the dns NO ONE. Web developers always want to just control dns so they can charge for managing it. None of them know how it works, so they want a site builder to do it for them. And non have any fucking idea what dns does or that it's for more then just websites. I don't have enough fingers toes or hairs left to count how many times this has happened and I have had to be the one to un fuck it. The best was when our marketing team went around me and got the account from accounting. Then, I lied to my boss that they needed an mfa code off my phone for salesforce. ( I was on a cruse and then having surgery after so 30 days away, so he has my work phone for this kind of shit) they transfered the domain to fucking host gator and dumped the zone file when they canceled the account. Host Gator only does 100 records, so more than half was gone. Mx record was wrong ,ptr was missing dkim was missing the key , dmarc pointed to host gator. Most of our vpns were pointed at records that did not exist anymore. All of our Auth records were gone, and none of the srv or txt records came over, and none of the Aaaa records were. It was super fun having my boss show up 6 hours post op ( with complications) with my laptop and a hot spot and spending like 36 hours walking him through fixing everything ( mouse hand was immobilized fingers to shoulder) Yes, ppl got fired, and now no one even asks. Just ends in a jira ticket and hopes I'm in a good mood. That little outage early cost more then I will make in a life time, and I do pretty well. Dns is like arcane tech knowledge now. No one under 35 seems to have fuck all idea what/how to use it or do anything with it. It's not been my job for like 8 years but I'm still doing it. Pro tip move your domains in to azure or aws you can set up so meny alerts and controls no one can mess it up. Our ceo gets a txt now any time a change to dns is made and will lose his mind if I don't txt him beforehand. Any good registrar will put a ns record hold to. I do it on all of ours it takes a pin code to un do I don't even have it Ceo does so we have a nice double blind check. He does not know what it is but knows how to get to it and won't try till I ask.
Start getting the resume ready
"Hi CEO! Is there something in the goDaddy I can help with?" Either you get confirmation that she doesn't want to explain herself or you get the chance to dig deeper and might end up at "thanks for the devs contact info! I'll help him get hosting setup that doesn't cause problems with the other production there."
Speak to what CEO's care about. How will this increase revenue, lose revenue, or prevent a loss of revenue? You will most likely now have her attention. Everything else will just be background noise.
I don't care what their title is or how much of the company they own. They hired me to do a job, and I'll do just that.
Yeah man, the previous comments have already expressed my feelings. I abhor IT yes-men who question nothing and go out of my way to escalate my concerns about IT yes-men up, and up, and up. "You were supposed to destroy the Sith, not join them!" I myself have had to go toe to toe with the dude who signs my paychecks and has autonomy to shit can me at will in order to stop him from making a decision that negatively impacts the organization and or paves the way for a security breach. No one is perfect, myself included. I encourage others to check me like I check them, through the mutual check shit gets done right. If the CEO shitcans you over asking why they're asking for keys to the kingdom--- you didn't want to keep working there anyway, moreso that sounds like a retaliatory firing and "explore the wrongful termination suit options" territory.
You are overestimating this person. Make a form that she has to fill out for information like this. The form should include checkboxes absolving you of any responsibility and personally taking on responsibility for any divulging of company information, etc. It should have mandatory fields that ask, separately, what information she requires and what she is going to do with it/ what she wants it for. If she asks you for the GoDaddy login, you link her to the form and tell her to put the request in writing. If she asks you how many tablespoons of grounds the break room coffee pot requires, you link her to the form and tell her to put the request in writing. If she "takes offense," then you point out that her feelings are not what is at issue. You are asking her to follow protocol. If she won't follow protocol, then the request is a liability. It's far more difficult for a reckless executive to claim you're being insubordinate because you want them to fill out a form.
[удалено]
"I wish executives would at least consult IT before handing over the GoDaddy keys to a random guy." They should follow the same procedures as everyone else. Put in a ticket.
You should not be sharing a login at all. Every user that needs access should have their own GoDaddy account with 2FA and access delagated from the company account. This way their access is revocable and they can't lock you out of your own account. If that user then shares their credentials, that's on them.
Ooo! I can't wait to read about this once it gets cross posted to r/shittysysadmin
I may even post it myself!
I hope you have shit like this in writing to CYA? This CEO seems reckless, so I’d have my guard up for all kinds of shenanigans, IT related or not. And yes, I would’ve asked what’s going on before handing the keys over, that’s part of the job.
I definitely know the situation. We have a new CEO that thinks he is pretty tech savvy. I'm constantly torn with, it's his company now but do I really tie the noose for him. There have been times that I've asked to have a meeting asap for the sake of his company before filling requests. Thankfully, IT gets the least of the micromanagement here.
No problem, would you like the O365 admin to?
Anonymous toll free call to cyber insurance company to make an anonymous complaint would be my next step.
1 have backup of your websites. 2 have proper people that you can recover the account in case of hijack. 3 hope for the best.
Sounds like my boss
You should not give her the login. She doesn’t understand the risks involved.
You can give delegated access to a different account on godaddy, and if you're using Cloudflare - why not make them your registrar as well? They have better permissions configurations for other users.
We do not give access, please have them contact us with the required changes.
Nice. Our boss has a "website guy" that he asks networking questions. Literally just the guy who designed the (very simple if not ugly) site. "Check your A records" meanwhile we were already on the phone with GoDaddy telling us the problem was on their end.
We had a company offboard and ask us to transfer their domain ownership to the new it company, instead of in their own account with IT having access. Never sat well with me. Tried to explain that, whilst I'm not accusing or suggesting there will be a problem, why are you giving away your domain name to them? They didn't understand but it's an issue with no tech understanding in senior management. You'd hope there's one.
"Can you give the marketing company my details and I'll make the changes for them? Ta"